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EVALUATION 


Thik  effort  is  part  of  a program,  the  goal  of  which  is  the  implementation 
of  a ecntrsl  repository  for  data  and  technical  information  for  computer 
software  technology  and  engineering.  This  center  will  provide  tools  and 
facilities  to  store,  analyze,  and  disaemlnate  authoritative  scientific  and 
technical  information  concerning  all  aspects  of  computer  software  and  the 
software  development  process. 

This  effort  fits  into  the  RADC  Technology  Plan,  particularly  RALC  TPO 
11,  Software  Sciences  Technology.  The  objective  was  to  investigate  areas 
relating  to  the  design,  implementation  and  operation  of  a central  repository 
for  software  data,  and  to  provide  a prelininary  design  Including  functional 
description  and  recommendations  for  a pilot  capability.  All  major  objectives 
of  this  program  were  successfully  addressed. 

These  results  in  concert  with  recommendations  from  a parallel  study  of 
software  data  collection  procedures  and  problems  are  being  incorporated  into 
RADC  plans  for  the  Implementation  of  a Data  and  Analysis  Center  for  Software. 
The  Software  Data  Collection  Study  was  conducted  by  System  Development  Corp 
and  is  reported  on  in  RADC -TR -76-32 9. 
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Introduction 


1.1  Report  Introduction 

This  report  presents  the  results  of  a study  to  develop  the  design  for 
a software  data  repository  to  collect,  analyze,  and  disseminate  software  de- 
velopment experience  information. 

1.2  Background 

Software  development  research  has  been  hampered  by  the  lack  of  data. 
Without  the  availability  of  historical  data  on  the  production  of  computer 
software,  we  lack  the  ability  to  predict  costs  of  future  development,  to 
determine  the  best  design,  development,  and  testing  methodologies,  and  the 
ability  to  effectively  measure  and  predict  the  reliability  of  computer  soft- 
ware systems. 

In  recognition  of  this  need,  RADC  contracted  for  this  study  to  establish 
the  specifications  for  a software  data  repository  for  data  and  information 
exchange.  This  repository  wilj.  serve  the  government/unlversity/industrlal 
community  as  the  focal  point  for  software  development  experience  information. 
Data  acquired  from  various  sources  will  be  scored  in  a centralized  data  base 
and  made  available  to  qualified  users,  analyses  will  be  performed  by  compu- 
ter software  engineers  and  statisticians,  and  results  will  be  produced  in 
the  form  of  publications  to  be  disseminated  to  repository  users. 

The  purposes  of  the  software  data  repository  are  to  upgrade  the  software 
development  process  through  Che  collection,  analysis  and  dissemination  of 
software  development  experience  information;  to  provide  a better  understand- 
ing of  Che  nature,  causes  and  effects  of  software  failures;  and  Co  determine 
those  factors  that  contribute  to  the  productivity  and  cost  of  computer  soft- 
ware development  and  maintenance. 

The  concept  of  a data  center  for  the  collection,  analysis  and  dissemina- 
tion of  information  in  a specialized  area  is  not  new.  As  early  as  1963,  in 
"The  Weinberg  Report"^^^  400  Information  Analysis  Centers  were  identified. 
This  panel  recognized  Che  need  for  centers  that  would  distill  technical  in- 
formation and  would  serve  more  than  disseminators  of  documents.  The  1970 
version  of  the  Directory  of  Federally  - sponsored  Centers  lists  110  centers 
(COSATI  Panel  6).  As  an  example  of  these  centers,  there  are  presently 
eight  DOD  Information  Analysis  Centers  which  are  administratively  managed 
and  funded  by  the  Defense  Supply  Agency  (DSA) . They  are  responsible  for  the 
collection,  evaluation,  analysis  and  dissemination  of  scientific  and  techni- 
cal information  to  the  scientists,  engineers  and  technicians  they  support. 
Among  Che  specialized  areas  they  deal  with  are  hardware  reliability,  metals 
and  ceramics,  mechanical  properties,  thermophyslcal  and  electronic  properties 
and  machinabillCy. 

The  generation  of  Che  design  for  the  software  data  repository  de- 
fined in  this  report  was  guided  by  the  IITRI  experience  in  developing  and 
operating  one  of  the  above  mentioned  DOD  Information  Analysis  Centers  - The 
Reliability  Analysis  Center  (RAC).lOlt  106-114 
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1.3  Report  OrgcDlxetlon 


Thlt  report  coatelne  eight  eections  end  three  eppendlccs. 

Section  1 contains  background  infomatlon  while  Section  2 providea  a func- 
tional definition  of  the  Software  Data  Repository  including  a discussion  of 
the  inputs,  processes,  and  outputs. 

The  requlreoents  for  processing  the  Input  data  and  documents  Is  presented 
In  Section  3 and  Section  4 contains  the  requirements  of  an  information  sys- 
tem to  manage  the  data  Including  a discussion  of  the  database  characteristics 
and  structure. 

Section  5 contains  discussions  on  data  collection  systems  that  automati- 
cally and  manually  collect  aoftware  data,  data  entry  hardware  devices  that 
are  available  on  the  market  for  consideration  as  tools  to  transform  hard 
copy  data  to  computer-readable  form,  alternative  information  system  and 
database  management  system  approaches,  and  alternative  approaches  for  auto- 
mating the  document  library. 

The  design  for  the  Software  Data^Repository , Including  the  devel- 
opment and  operation  of  a pilot  facility  and  the  expansion  to  a fully  opera- 
tional center,  is  presented  in  Section  6.  Section  7 contains  a summary  of 
the  report  recommendations  and  a bibliography  Is  presented  In  Section  8. 

Appendix  A contains  a glossary  of  database  management  terms.  Presented 
in  Appendix  B are  the  results  of  a survey  of  the  software  development  com- 
munity to  determine  their  interest  in  the  repository.  Appendix  C contains 
definitions  of  software  engineering  terms  as  they  appear  in  the  literature. 


2. 


Definition  of  Che  Software  Data  Repository 


2.1  Introduction 

This  section  presents  and  describes  Che  functional  model  for  Che  Software 
Data  Repository.  In  pursuit  of  this  model  definition,  the  following  sub- 
casks were  performed: 

- Review  of  Che  software  engineering  literature 

- Survey  of  Che  software  development  community 

- Visits  CO  software  developers  and  attendance  at  software  engineering 

conferences 

- Meetings  with  RADC  engineers 

- Review  of  Che  literature  on  data  center  operations 

- Observing  the  operation  of  the  Reliability  Analysis  Center 

2.2  Functional  Model 

The  Hierarchy  Input  Process  Output  (HIPO)  chart  in  Figure  2.1  provides  a 
graphical  representation  of  Che  functional  model  of  the  Software  Data  Repos- 
itory (SDR).  The  principal  inputs  to  the  SDR  include  production/development 
reports  Chat  contain  software  experience  data  and  documents  that  contain 
cextua?  information  on  Che  software  development  process.  The  data  and  docu- 
ments form  Che  basis  for  a document  library  and  Che  two  computer  databases 
Chat  serve  as  inputs  for  subsequent  updating  and  processing. 

The  major  SDR  processes  include  the  acquisition  of  data  and  documents, 

Che  evaluation  and  summarization  of  these  inputs,  the  production  of  reports 
and  data  subsets,  Che  performance  of  engineering  analysis,  data  analysis, 
and  software  model  development. 

The  major  outputs  are  the  updated  databases  and  document  library,  data 
subsets  for  modeling  efforts,  published  reports  containing  results  of  analy- 
sis, and  answers  to  inquiries  and  consulting  actions. 

As  Che  document  library  and  the  two  major  databases  form  the  heart  of  Che 
repository,  Che  information  content  of  each  is  first  discussed. 

2.3  Information  Concent 

There  are  two  major  classes  of  information  chat  form  the  inputs  to  the  SDR. 
The  first  class  consists  of  productlon/development  information  relating  to 
Che  developmental  characteristics  of  producing  and  monitoring  computer  soft- 
ware. The  second  class  consists  of  textual  or  expository  ioformation  relat- 
ing Co  software  development.  From  these  documents  Che  Document  Description 
Database,  the  Historical  Database  and  Che  Complementary  Database  are  generat- 
ed (Figure  2.2). 

The  Document  Description  Database  contains  Che  information  needed  to  re- 
trieve Che  document  according  to  the  users'  need.  This  database,  plus  the 
source  documents,  form  the  basis  for  a document  library. 
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SOFTWARE  DATA  REPOSITORY 


FIr.  2.1  Functional  Model 


Fig.  2.2  Information  Content 


Tht  Hlatorlc«l  Database  contains  auamarlxcd  Infocmatlon  obtained  iron  the 
productlon/dcvelopnent  reports.  The  types  of  Infomatlon  contained  in  this 
database  arc  productivity • cost,  anvironsent,  and  sumaarised  teat  and  error 
data. 

The  Coaplaacntary  Database  contains  detailed  testing,  error,  and  change 
information  obtained  from  the  production/development  reports. 

2.3.1  Source  Data/Doctacncs 

There  are  two  major  classes  of  information  that  form  the  inputs  to  the 
SDR.  They  are: 

1.  Production/development  reports  related  to  the  developmental  character- 
istics of  producing  and  maintaining  computer  software.  This  data  may  be 
in  hard  copy  or  computer-readable  form  and  provides  experience  data  from 
the  following  categories: 

. Project  Description  Reports 
. System  Development  Logs 
. Configuration  Management  Reports 
. Program  Support  Library  Outputs 
. Problem  Reports/Correction  Reports 
. Testing  Summary  Reports 
. Automated  Testing  System  Outputs 
. Operatlon/Maintcnance  Reports 
. Engineering  Change  Reports 

2.  Textual  or  expository  information  related  to  software  development. 

These  documents  are  from  the  following  categories: 

. Journal  Articles/Tcchnlcal  Papers 
. Specifications/Standards 
. Symposia  and  Conference  Proceedings 
. Bibliographies,  Surveys,  Reviews 
. Handbooks 

Research  and  Development  Reports 
. Unpublished  Technical  Papers 

2.3.2  Document  Description  Database 

An  entity  represents  an  object  or  event  in  the  system  being  modeled. 
This  database  is  made  up  of  two  entities. 

. The  Document  Descriptor  Entity  - information  pertinent  to  describing 
the  characteristics  of  the  data  or  the  document. 

. The  Document/Data  Cross  Reference  Entity  - information  necessary  to 
retrieve  the  production/development  data  stored  in  the  other  two 

databases . 
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2.3.3  Historical  Database 

The  Historical  Database  contains  summarized  information  from  the  pro- 
duction/development reports.  The  data  is  stored  at  the  project,  system 
and  module  levels  for  each  phase  in  the  software  life  cycle. 

As  used  within  this  report,  a project  consists  of  one  or  more  systems 
and  provides  a solution  to  a problem.  A system  consists  of  one  or  more 
modules  and  provides  a meaningful  software  product  to  the  user.  A module 
is  a discrete  identifiable  set  of  instructions  handled  as  a unit  by  an 
assembler,  compiler  or  loader. 

The  software  life  cycle  consists  of  six  phases  including  the  conceptu- 
al, requirement,  design.  Implementation,  test  and  operation  phases. 

1.  Conceptual  Phase 

Problem  statement  definition 
Preliminary  systems  analysis 
Alternative  solution  categories 

2.  Requirement  Phase 

Project  objectives 

System  specifications 

Request  for  Proposal  (RFP)  preparation 

Proposal  preparation  and  evaluation 

3.  Design  Phase 

Software  component  definitions 
Interface  and  data  definitions 
Requirement  compliance 

4.  Implementation  Phase 

Program  code  production 
Unit  testing 

5.  Test  Phase 

System  integration  of  the  software  components 
System  and  acceptance  tests 
Requirement  compliance 

6.  Operation  Phase 

System  use 
System  maintenance 
Detection  and  correction  of  errors 

Modification  to  add  capabilities  and/or  improve  i erf ormance 
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The  Bletorlcal  Databaae  1*  Bade  up  of  clx  entitles  (Figure  2.3)  end 
the  contents  of  each  is  briefly  described  below. 

The  Project /Cooponent  Descriptions  Entity  • descriptive  end  structural 
Inforastlon  for  the  project  coaponents;  experience  infonatlon  on  per- 
sonnel assigned  to  the  project;  and  Information  on  the  computer  system 
used. 

. The  Environment  Entity  - Information,  for  each  component  and  phase,  on 
the  types  of  personnel  and  computer  used,  contract  type,  applicable 
standards  and  collection  procedures,  and  project  constraints  and  pri- 
orities. 

. The  Technology  Entity  - information  on  the  various  management,  develop- 
ment and  testing  philosophies,  tools  and  techniques  used  for  each  com- 
ponent and  phase. 

. The  Resource  Utilization  Entity  - the  calendar  time  spent,  the  amount 
of  time  for  each  personnel  category  and  for  the  computer,  and  the  costs 
for  computer,  personnel,  travel  and  miscellaneous  Items. 

. The  Production  Entity  - suenarized  information,  for  each  component  and 
phase,  on  the  number  of  pages  of  documentation  produced;  the  number 
and  types  of  changes  made,  tests  run,  and  errors  uncovered  and  cor- 
rected. 

. The  Software  Characteristics  Entity  - the  software  produced  in  terms 
of  quality,  complexity,  stress  applied  and  software  constituents. 

2.3.4  Complementary  Database 

The  Complementary  Database  contains  detailed  information  from  the  pro- 
duction/dcvelopment  reports.  As  in  the  Historical  Database,  the  data  is 
scored  at  the  project,  system  and  module  levels  for  each  phase  in  the 
software  life  cycle. 

The  Complementary  Database  Is  made  up  of  three  entities  and  is  expected 
to  grow  as  special  collections  on  various  projects  arc  made  (Figure  2.4). 

The  first  entity  Is  the  Project /Component  Descriptions  Entfry  as  dis- 
cussed tinder  the  Historical  Database.  If  this  Information  Is  recorded 
for  the  Historical  Database,  it  can  be  integrated  into  the  Complementary 
Database.  The  ocher  two  entities  are  the  Test  Results  Entity  and  the 
Enginearing  Change  Entity. 

. The  Test  Results  Entity  - detailed  information  on  the  characteristics 
of  each  test  run  including  stress  applied,  error  category,  aeverity 
and  correction  type,  resources  used,  and  affected  areas. 


. The  Engineering  Change  Entity  - detailed  information  on  the  character- 
istics of  each  enginearing  change  including  change  type,  calendar  time 
and  resources  used. 
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The  information  in  this  dstsbsse  is  sxpseted  to  be  dynaaic  to  support 
various  rasaarch  efforts  that  demand  detailed  experience  data.  As  special 
needs  are  Identified  such  as  performing  research  in  error  analysis,  amin> 
tainability,  complexity  measures,  and  program  constructs,  additional  en- 
tities will  be  provided  dependent  upon  data  availability.  Also,  obsolete 
entities  vill  be  deleted  as  data  requirements  change. 

2.4  Process  Overview 

Major  processing  elements  for  the  repository  are  briefly  described  in  the 
following  paragraphs. 

2.4.1  Data/Dorument  Acquisition 

A concerted  and  continual  acquisition  effort  is  required  with  full  co- 
operation of  the  many  sources  that  generate  information  and  data  that  may 
be  acquired  through  direct  solicitation,  voluntary  contributions  and  in- 
corporation of  data  requirements  into  software  development  contracts.  The 
major  sources  of  data  are  military  systems  program  offices,  other  govern- 
ment agencies,  project  contractors,  computer  software  developers,  indepen- 
dent R4D  organizations,  academic  coBsaunity,  symposia,  conference,  seminars, 
and  other  open  literature. 

2.4.2  Input  Processing 

It  is  required  that  the  Incoming  data  and  documents  be  thoroughly  re- 
viewed for  relevancy,  validity,  accuracy  and  completeness  prior  to  accep- 
tance. This  review  is  conducted  by  computer  professionals  with  technical 
expertise  in  software  engineering.  An  accepted  document  is  cataloged  in 
the  library  for  subsequent  retrieval.  If  this  input  contains  production/ 
development  data,  it  is  summarized  and  transformed  into  the  SDR  formats 
and  then  entered  into  the  corresponding  database(s). 

2.4.3  Report  Production  and  Engineering  Analysis 

The  report  production  for  the  SDR  Implies  producing  subsets  of  the  data 
in  both  computer  readable  and  hard  copy  form,  state-of-the-art  reports  on 
designing,  developing,  and  maintaining  computer  software,  and  the  results 
of  software  modeling  and  other  analysis  efforts. 

! The  types  of  analysis  include  evaluation  and  refinement  of  software 

; reliability  and  maintainability  models,  regression  analysis  on  the  factors 

^ that  affect  productivity  and  cost,  and  other  data  analysis  techniques  such 

as  distribution  fitting,  calculation  of  means,  averages,  correlations,  and 
t frequency  distributions, 

j- 

I 2.5  Output  Types 

! 

The  major  output  types  and  their  uses  are  briefly  described  in  the  follow- 
ing paragraphs: 

. Database  - access  to  the  database  for  searching  and/or  copying  the  con- 
tents of  the  databases. 
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. Data  Subscription  - computer  readable  copy  of  Che  database (or  a subset 
of  the  database)  and  regular  updating  service. 

. Data  compendlums  - publications  that  provide  summary  Information  on  Che 
database  holdings. 

. Handbooks  - publications  that  provide  practical  guidelines  and  procedures 
for  producing  and  maintaining  computer  software. 

. Scate-of-che-Arc  Reports  - publications  chat  provide  Che  latest,  authori- 
tative Information  on  software  development  techniques. 

. Technical  Monographs  - In-depth  reports  on  a specialized  software  develop- 
ment area  such  as  evaluation  and  validation  of  software  reliability  models. 

. tnquery  and  Consulting  Services  - these  services  provide  the  user  with 
knowledgeable  assistance  In  Che  solution  of  a problem. 

. Document  Library  - access  to  the  documents  within  the  library  and/or  dis- 
tribution of  copies  of  the  documents. 

. Bibliography  - a publication  including  references,  bibliographic  citations, 
and  indices  for  user  search. 


I 

I 


I 
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3.  Input  Processing  Requirements 


3.1  Introduction 

One  of  the  most  critical  and  time  consuming  functions  of  the  repository 
is  that  of  processing  the  input  to  assure  that  the  information  is  valid, 
meaningful  and  easily  retrievable.  The  purpose  of  this  section  is  to  de- 
scribe the  methods  required  to  process  the  input.  Figure  3.1  contains  an 
overall  flow  of  the  input  processing. 

The  initial  processing  of  the  production/development  reports  and  the  tex- 
tural documents  is  similar.  Both  types  of  Inputs  are  evaluated,  indexed, 
and  stored  for  future  reference.  If  the  input  includes  production/develop- 
ment reports,  the  data  from  these  reports  is  transformed  and  loaded  into  the 
Historical  and  Complementary  Databases. 

3.2  Evaluation 

The  first  task  is  to  evaluate  all  incoming  data  and  documents  by  careful- 
ly reviewing  the  input  to  determine  if  the  information  is  valid  and  is  rele- 
vant Co  Che  software  development  process.  At  this  point,  it  is  either 
rejected  for  Inclusion  into  Che  SDR  or  accepced  for  furcher  processing. 

Each  accepCed  document  is  matched  against  the  current  source  documents  for 
duplication.  If  a duplication  exists,  the  duplicate  document  is  stored  with 
Che  original  and  no  furcher  processing  is  required.  If  a duplication  does 
not  exist,  the  accepted  document  is  Chen  indexed. 

This  evaluation  process  also  takes  into  account  the  corporate  sensitivity 
of  the  data  or  the  documents.  A determination  is  made  at  the  time  of  the 
levels  of  security  that  must  be  applied  to  this  information. 

3.3  Indexing 

Indexing  is  the  Cask  of  assigning  descriptive  terms  to  the  input.  This 
is  an  important  process  since  the  access  to  this  information  is  through  the 
index  terms  assigned. 

The  Document  Description  Form  (Figure  3.2}  is  completed  during  this  in- 
dexing task  to  provide  a consistent  recording  of  all  indexing  information. 

The  indexing  information  for  the  reports  and  dociiments  is  grouped  into  three 

classes: 

1)  location  term 

2)  bibliographic  terms 

3)  subject  terms  or  keywords 

The  location  term  assigned  to  any  report  or  document  is  its  accession 
number.  This  accession  number  is  unique  for  each  report  or  document  and 
serves  two  purposes: 

1)  it  is  the  primary  means  of  locating  the  report  or  document  within  the 
library 

2)  it  is  ths  primary  maans  of  cross-referencing  the  report  or  document  for 
retrieval 
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The  bibliographical  tanss  Include  title,  peraonal  and  corporate  author(s), 
publication  nane,  volume  number,  pagination,  publication  date  and  document 
type.  For  government  reporta,  report  number,  contract  number  and  contract 
datea  are  alao  included. 

Subject  terms  (keywords)  describe  the  textual  and  data  content  of  the  re- 
port or  document.  This  task  of  assigning  keywords  Is  the  most  time-consuming 
and  critical  procedure  in  Indexing.  The  user  depends  upon  the  document  being 
correctly  classified.  If  Incorrect  keywords  are  Inadvertently  assigned,  that 
information  is  less  than  optimally  suited  for  user  needs. 

An  Indexer  reviews  the  foreword,  abstract,  table  of  contents,  data  defini- 
tions, report  summary,  and  conclusions  of  each  document  and  produces  a list 
of  terms  that  describe  the  content  of  the  document.  It  is  very  Important 
that  the  Indexer  be  as  consistent  as  possible  and  call  the  same  concept  by 
the  same  term  everytime  It  appears.  To  facilitate  this  requirement,  a 
thesaurus  is  used. 

A thesaurus  provides  both  the  Indexer  and  the  retriever  with  a list  of 
permitted  terms.  It  contains  definitions  of  the  terms,  cross  references  to 
synon>’ms  and  related  terms,  and  cross  reference  to  broader  and  narrower  terms. 

The  following  Is  a set  of  guidelines  for  selecting  subject  terms: 

. Express  all  subject  terms  as  nouns 

. When  citing  terms  describing  processes,  use  the  gerundive  form  of  the 
verb.  For  example:  debug  becomes  debugging 

. Use  singular  forms  when  citing; 

- Specific  material  terms 

e.g.,  flow  chart,  schedule,  disc 

- Specific  terms  representing  properties,  conditions  or  character- 
istics 

e.g.,  reliability,  availability,  predictability,  productivity 

- Terms  describing  a specific  process 

e.g.,  debugging,  verifying,  program  structuring 

- Proper  names 

e.g.,  Shooman  Model 

- Terms  describing  disciplines,  fields,  subject  areas 
e.g.,  computer  science,  automated  testing 

. Use  plural  forms  when  citing 

- Generic  terms 

••g.»  operating  systems,  design  techniques,  costs 

- Terms  describing  equipment  or  devices 
e.g.,  computers,  compilers 

- Terms  describing  events  or  occurrences 
e.g.,  failures.  Interactions 

. Use  terms  that  arc  recognized  in  the  software  development  community, 
are  usamblguous,  and  do  not  occur  too  frequently  in  the  literature. 
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3.4  Updating 


Updating  la  the  process  of  storing  the  source  Inputs  In  the  document  li- 
brary and  entering  the  Information  from  the  Document  Description  Form  into 
the  Document  Description  Database. 

The  source  Inputs  are  labeled  and  stored  by  accession  number  with  a copy 
of  the  Document  Description  Form  attached  to  the  document.  The  particular 
storage  media  Is  dependent  upon  the  document  form.  Hard  copy  documents  are 
stored  In  file  folders  In  filing  cabinets,  magnetic  tapes  are  stored  In  tape 
racks,  and  punched  computer  cards  are  stored  In  card  files. 

The  Information  from  the  Document  Description  Form  Is  transformed  Into 
computer-readable  form  and  entered  Into  the  Document  Description  Database. 

The  processing  for  the  textual  documents  Is  now  complete.  The  production/ 
development  reports  require  further  processing. 

3.3  Analysis  of  Data  Form  and  Content 

Since  the  production/development  data  is  destined  for  the  Historical  and 
Complementary  Databases,  an  analysis  is  made  of  the  form  and  concent  of  the 
data  to  determine  subsequent  processing  procedures.  The  data  is  examined 
for  Incompatibilities  with  the  requirements  for  the  Historical  and  Comple- 
mentary Databases.  These  incompatibilities  take  the  form  of  inconsistent 
coding  values  and  field  lengths,  incompatible  summarization  levels,  and  dis- 
tribution of  data  values. 

3.6  Data  Translation 

The  production/development  data  is  transformed  to  the  formats  suitable 
for  the  SDR  databases.  This  translation  is  performed  using  manual  proce- 
dures and  computer  programs. 

Using  manual  procedures,  this  process  Includes  manually  transforming  the 
data  into  the  SDR  formats  and  producing  a computer  readable  copy  of  the 
transformed  data. 

Using  computer  programs,  this  process  includes  defining  and  reading  the 
source  data,  transla*’ l..g  the  data  according  to  the  SDR  data  requirements  and 
outpuclng  the  transformed  data.  The  extent  of  the  translation  is  dependent 
upon  Che  incompatibilities  uncovered  during  Che  analysis  of  the  data. 

3.7  Database  Updating 

This  process  includes  entering  Che  transformed  data  into  Che  Historical 
and  Complementary  Databases.  During  Che  updating  process,  Che  data  is  edited 
for  incorrect  coding  values,  inconsistent  records,  and  duplication  of  infor- 
mation. If  needed,  the  data  is  corrected  and  reloaded  into  the  database. 


3-5 


p 


A.  The  SDR  Information  System 
A.l  Introduction 

An  Information  system  for  the  SDR  requires  a computer  hardvare/sof tvare 
system  that  provides  processing  capabilities  for  the  SDR  and  a database 
management  system  that  provides  automated  tools  for  managing  the  data. 


This  section  presents  a general  discussion  on  the  requirements  of  the 
hardvare/sof tware  system  including  the  capabilities  of  the  ARFA  network,  the 
general  requirements  of  security  procedures,  a presentation  on  the  require- 
ments of  a database  management  system  (DBMS)  to  manage  the  SDR  databases, 
and  a discussion  of  the  characteristics  of  the  SDR  databases. 

A. 2 Requirements  Summary 

The  basic  feature  requirements  of  an  information  system  for  the  SDR  are 
listed  in  Ta*'Ie  A.i  Each  feature  has  two  columns  associated  with  it.  The 
first  column  illustrates  the  need  of  a feature  for  a pilot  facility,  the 
second  column  illustrates  the  need  for  a fully  operational  center.  The  pri- 
ority need  is  established  according  to  the  following  levels; 

M Mandatory  This  feature  must  be  available. 

D Desirable  It  would  be  desirable  to  have  this  feature,  but  if 

not  available,  a suitable  alternative  can  be  found. 

0 Optional  It  would  also  be  desirable  to  have  this  feature  but 

if  it  is  not  available,  normal  operation  would  not 
be  impaired. 

N Mot  needed  Even  if  this  option  were  available,  it  would  not  be 

used . 

Blank  The  requirement  level  has  not  been  established. 

Each  of  these  features  are  discussed  in  the  following  subsections.  A glos- 
sary of  terms  is  presented  in  Appendix  A for  those  readers  who  are  not 
familiar  with  database  terminology. 

A. 3 Hardvare/Sof tvare  Systems 

A hardvare/sof tvare  system  to  be  used  by  the  repository  must  provide  fa- 
cilities for  online  and  batch  processing  of  standard  and  ad  hoc  retrievals, 
batch  processing  for  standard  and  any  ad  hoc  reports,  onllne/batch  proces- 
sing for  retrievals  that  result  in  lengthy  answers,  batch  processing  for  the 
maintenance  and  updating  of  the  database.  The  system  must  be  able  to  inter- 
face retrieval  results  and  report  generation  with 'application  programs. 

There  are  at  least  three  options  that  fulfill  these  general  requirements. 
The  first  is  to  utilize  a medium  to  large  scale  computer  with  its  attendant 
software  and  full  range  of  peripherals,  such  as  found  in  many  computing  cen- 
ters (i.e.,  RADC  computer  center).  The  second  is  to  purchase  services  from 
outside  vendors.  A third  option  is  to  purchase  a mini-computer  and  perform 
all  of  the  processing  functions  In-house. 
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Table  4.1 

Feature  Requircoents 


Feature  Requircoents 

Required 

Feature 

Pilot 

Full 

1.  Hardware/Softvare  Systco 

RADC  Cooputcr  Center 

M 

Dedicated  Mini-' coopu ter 

N 

Outside  Cooputcr  Service 

N 

Interface  to  the  ARPAKET 

0 

D 

2.  Mode  of  Operation 

Batch  Retrieval 

0 

0 

Batch  Maintenance 

M 

M 

Online  Retrieval 

0 

M 

Online  Maintenance 

0 

0 

Online/Batch  Retrieval 

M 

D 

Online/Batch  Maintenance 

D 

D 

3.  Users 

Data  Adoinistrator 

D 

M 

Applications  Prograaoer 

0 

M 

Nonprograsner 

M 

M 

Parametric  User 

D 

M 

4.  Data  Loading  and  Maintenance 

Self-Contained 

M 

M 

Host  Language 

D 

M 

Foreign  Files 

M 

K 

Input  Verification 

D 

D 

Reorganization 

D 

K 

Restart  and  Recovery 

D 

K 

Data  Translation 

D 

hi 

3.  Retrieval  and  Report  Generation 

Self-Contained 

M 

K 

Interface  to  User  Progrsos 

D 

K 

Data  Subsets 

K 

M 

Multi-Attribute  Qualification 

M 

hi 

Multi-Attribute  Sorting 

K 

hi 

Multi-Users 

0 

hi 

6.  User  Programs 

} Data  Manipulation  Language 

D 

1 Data  Files 

M 

M 

7.  Database  Characterization  and  Structure 

Data  Description  Language 

1 Schcaa-SubscheDa 

Data  Structuring 

M 

D 

M 

M 

M 

1 Data  Directory 

D 

” 1 

Table  4.1 

Feature  Requirements  (cont’d) 


Feature 


Required 


Pilot  Full 


8.  Security 
Method 

Administrative 

Hardware 

Operating  System 
DBMS 

Technique 
Read  Access 
Write  Access 
Log-on 
Password 
Level 
Database 
File 
Field 


M 

0 

M 

D 

M 

M 

M 

M 

D 

D 

D 


M 

D 

M 

M 
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Any  one  of  chest  onvlronments  could  be  part  of  a computer  network  (such 
as  the  ARFA  natvork).  A computer  network  would  facilitate  direct  access  to 
a database  and/or  the  application  programs  by  a multitude  of  users. 

4.3.1  ARPA  Network 

The  ARPA  network  requirements  for  the  SDR  are  to  provide  Che  facili- 
ties for  distributing  data  subsets  and  receiving  foreign  files,  and  to 
provide  direct  access  to  the  database  by  qualified  users.  RADC  has  ac- 
cess  to  the  ARPA  network  through  the  Multlcs  Operating  System. 

4. 3. 1.1  Introduction 

The  ARPA  Network  (ARPANET)  has  been  operational  since  1969  and  Is 
composed  of  over  85  host  computers  distributed  over  60  different  loca- 
tions in  the  United  States.  The  geography  and  topology  of  the  network 
are  illustrated  in  Figures  4.1  and  4.2  respectively.^^' 

The  motivations  for  the  network  were  to  establish  computer  resource 
sharing,  to  assist  in  contractor  communication,  and  to  demonstrate  the 
feasibility  of  packet  switching  technology. 

This  network  was  originally  conceived  and  funded  by  the  Advanced 
Research  Projects  Agency  (ARPA)  of  the  Department  of  Defense.  The 
Defense  Communication  Agency  (DCA)  has  the  primary  responsibility  now 
and  each  user  Is  assessed  for  their  network  usage.  Last  year  RADC  was 
assessed  $60,000.  This  year  It  is  expected  to  be  $72,000.  Bolt  Bera- 
nek  and  Newman  (BBS)  Is  the  primary  contractor  for  the  development  and 
operation  of  the  network.  The  KIT  Laboratory  of  Computer  Sciences  Is 
presently  under  contract  to  develop  ARPANET  capabilities  for  the  KULTICS 
operating  system  on  the  Honeywell  6180  Computer  system. 

The  major  attributes  of  the  ARPANET  are  listed  below. 

. A distributed  store  - and  - forward  network  of  heterogeneous  compu- 
ter system  (hosts). 

. Network  Control  Programs  (NCP)  are  the  software  interfaces  for  con- 
necting hosts. 

. The  hardware  Interfaces  have  the  characteristics  of  either  a channel 
or  a communications  line. 

. An  Interface  Message  Processor  (IMP)  is  a specially  modified  Honey- 
well 316  or  516  processor  which  serves  at  the  communications  compu- 
ter in  the  network. 

. A Plurlbus  IMP  Is  a multiprocessor  IMP  assembled  by  BBK  using  micro- 
processors. 

. Host  to  host  messages  are  passed  from  the  host  to  Its  IMP,  broken 
Into  packets,  and  sent  to  their  destination. 

. The  route  of  any  given  packet  Is  not  established  in  advance  and 
several  packets  of  a message  may  follow  different  routes. 

. The  destination  IMP  reassembles  the  message  and  delivers  it  to  the 
proper  host. 

. Three  Protocol  Levels  have  been  astabllshed  for  communication: 
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- Level  1 governs  logical  exchange  of  Information  between  host  and 
IMP. 

- Level  2 governs  logical  exchange  of  Information  between  NCPs  In 
communicating  hosts. 

- Level  3 governs  communications  occurring  between  processes  In  the 
host  machines.  Examples  of  this  level  Include  the  data  transfer 
protocol,  Che  file  transfer  protocol,  and  Che  special  TELNET  Proto- 
col. The  TELNET  defines  a network  virtual  terminal  and  permits  all 
terminals  on  the  network  to  provide  a similar  Interface  to  proces- 
ses In  a separate  host. 

. A Terminal  IMP  (TIP)  Is  an  IMP  which  Is  augmented  by  additional  mem- 
ory and  a multiline  controller.  The  TIP  contains  a network  control 
program  and  a TELNET  program  to  permit  terminals  to  access  Che  net- 
work directly 

4. 3. 1.2.  RADC  - ARPANET  Accessibility 

RADC  is  presently  a TIP  user  on  the  ARPANET  through  theMulclcs  Op- 
erating System  (Figure  4.3).  Below  Is  a brief  summary  of  the  status 

of  ARPANET  accessibility  by  RADC: 

. The  lower  level  protocols  (Levels  1 and  2)  are  well  established  and 
would  be  of  little  concern  to  the  SDR. 

. The  file  transfer  protocol  (FTP)  is  also  well  established.  The 
transfer  parameters  Include  type  (ASCII,  BCD,  etc.),  mode  (scream, 
block),  and  structure  (file,  record).  This  protocol  is  used  exten- 
sively for  transferring  messages  (mall  facility).  In  addition  to 
this  protocol,  there  Is  a need  for  data  transformation  software  (as 
discussed  In  a previous  section). 

. The  ocher  higher  level  protocols  are  not  as  well  established  and 
Che  user  must  learn  the  host  dependent  commands  and  conventions 
(log-on  procedures,  job  control  language,  file  naming  conventions, 
etc.).  In  addition,  Che  quality  of  documentation  and  assistance 
availability  varies  from  site  to  site.  There  is  a need  for  a sys- 
tem control  language  to  provide  a common  means  for  users  and  appli- 
cation programs  to  access  the  resources  needed.  Independent  of 
their  network  location. 

. The  communication  system  Is  highly  reliable. 

. The  reliability  of  the  host  system  varies  from  site  to  site. 

. Backup  processing  capability  Is  left  to  the  user. 

. Flic  security  Is  the  responsibility  of  the  Individual  host. 

4.3.2  SDR  Options  for  Utilizing  the  ARPANET 

The  ARPANET  can  be  used  for  the  repository  at  RADC  to  transfer  subsets 
of  the  production/development  data  to  qualified  users,  and/or  to  receive 
productlon/development  data  from  the  software  developer. 

A higher  level  of  usage  of  the  ARPANET  la  to  allow  qualified  users  a 
, retrieval  capability  to  provide  them  with  access  to  the  Document  Description, 
Historical  and  Complementary  Databases.  In  addition  the  repository  person- 
nel may  retrlava  productlon/development  data  from  the  developers'  database 
to  determine  applicability  to  the  repository  before  actually  receiving  the 
data. 
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4.4  Security 

As  some  of  the  product /development  reports  will  Include  corporate  sensi- 
tive productivity  and  cost  data,  procedures  and  mechanisms  must  be  estab- 
lished and  utilized  to  prevent  unauthorized  access  to  this  sensitive  data. 

In  addition,  there  must  be  security  provisions  to  safeguard  against  the  acci- 
dental distructlon  of  data  and  the  disruption  of  user  services. 

The  SDR  security  involves  the  computer  system  Including  its  attendent 
hardware  and  software  components,  the  repository  users,  the  repository  and 
computer  site  personnel,  the  physical  environment,  and  the  relationships  and 
interactions  between  all  of  the  above  (Figure  4.4). 

Backup  procedures  such  as  restart  and  recovery  can  be  incorporated  to  pro- 
vide some  level  of  data  Integrity.  Also  editing  and  validation  can  be  accom- 
plished to  prevent  the  invalid  entry  of  data. 

Adequate  technological  safeguards  to  provide  protection  from  deliberate 
misuse  or  access  to  the  syszem  are  not  now  available  in  current  computer 
systems. 305  The  use  of  passwords  for  access  to  the  operating  system  and 
the  database  are  available  out  can  be  penetrated.  These  mechanisms  will  be 
used,  but,  at  least  initially,  the  sensitive  data  submitted  to  the  SDR  will 
physically  be  stored  for  manual  retrieval  rather  than  to  make  it  available 
through  the  computer  system. 


Sensitive  data  is  that  data  which  has  been  submitted  to  the  repository 
that  the  submitter  considers  sensitive.  That  is,  the  Judgement  of  sensitiv- 
ity will  be  left  up  to  the  contributor.  It  is  expected  that  the  sensitive 
data  may  consist  of  the  developers'  affiliation,  project  designations,  pro- 
ductivity data,  and  cost  data. 

4.5  Database  Management  System  Requirements 

Shown  in  Figure  4.5  are  the  major  database  management  system  functions 
that  must  be  considered  for  the  SDR.  3-^3.  Each  of  these  functions  is  discussed 
below  including  a discussion  of  the  types  of  users  of  a database  management 
system. 

4.5.1  Users 

206 

The  Conference  on  Data  Systems  Language  Systems  Committee  (CODASYL) 
defines  four  levels  of  users  of  database  management  systems.  These  arc 
data  administrator,  applications  programmer,  nonprogrammer  and  the  para- 
metric user.  The  data  administrator  is  the  one  person  who  is  responsible 
for  the  creation,  restructuring,  security,  etc.  of  the  database. 

The  next  level  is  the  applications  programmer.  These  persons  are  com- 
puter professionals  who  are  well  versed  in  the  current  practices  of  data 
processing.  They  are  responsible  for  writing  application  programs,  pro- 
cessing difficult  queries,  generating  reports,  etc. 
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Environment 


The  next  tvo  categories  (eonetlttes  tensed  the  general  user)  are  the 
oonprograasser  and  the  paraaetrlc  user.  The  nonprograssser  is  typically  a 
person  vho  is  knowledgeable  in  the  functions  of  the  organization  but  is 
not  necessarily  a computer  professional.  The  nonprogrammer  will  most  of- 
ten operate  in  an  online  environment  and  answer  user  queries.  For  the 
software  data  repository  it  is  assumed  that  the  "nonprograaner"  is  famil- 
iar with  the  software  engineering  field  but  queries  the  database  in  a 
nonprocedural  way  without  needing  to  know  the  structure  of  the  data.  It 
la  expected  that  all  professional  members  of  the  staff  of  the  SDR  will 
query  the  database  in  this  way  at  some  time  or  other.  Including  management. 

The  parametric  user  is  not  skilled  in  the  use  of  the  computer,  but  does 
have  the  knowledge  required  to  invoke  predefined  transactions  (i.e.,  sup- 
port personnel).  This  support  function  Includes  the  important  task  of 
data  entry. 

4.5.2  Data  Loading  and  Maintenance 

The  loading  of  data  is  a tedious  and  time-consuming  task  and,  as  such, 
offers  great  opportunity  for  error.  It  is  thus  extremely  important  that 
the  facility  for  input  verification,  and  error  detection  and  correction 
be  provided. 

As  the  input  data  will  be  arriving  from  many  sources,  the  data  must  be 
made  compatible  with  the  SDR  database.  There  is  need  for  the  capabilities 
to  read  foreign  files  and  translate  the  data. 

There  are  basically  three  categories  of  data  maintenance.  These  are 
edd/delete  records,  add/delete/change  values  in  a record,  and  a restruc- 
turing of  the  database.  The  tvo  modes  that  would  be  required  are  online/ 
batch  and  batch.  With  these  tvo  modes,  data  could  be  entered  online  and 
later  processed  in  batch  or  entered  in  the  batch  mode. 

There  is  an  additional  requirement  for  the  capabilities  of  restart  and 
recovery.  Procedures  for  capturing  information  about  changes  to  the  data- 
bases, audit  trails,  database  dumps,  roll  back  and  roll  forward  on  a 
transaction-by-transactlon  basis  are  required.  These  techniques  are  used 
to  restore  the  database  to  a prior  state  after  an  error  on  the  part  of  the 
user,  operator,  program,  or  malfunctioning  hardware. 

4.5.3  Retrieval  and  Report  Generation 

The  major  requirements  of  a retrieval  and  report  generation  subsystem 
are  to  provide  the  capability  of  qualifying  a subset  of  the  repository 
database  on  all  attributes  (and  any  combination  thereof),  sorting  and/or 
formatting  this  subset,  interfacing  this  subset  to  user  programs,  or 
printing  this  subset  directly  to  the  requesting  computer  terminal.  Since 
some  of  the  users  of  this  system  will  be  nonprogrammers,  there  is  a re- 
quirement for  online  tutorial  material  to  aid  in  retrieval  specifications. 

It  would  also  be  highly  deslreablc  to  have  feedback  on  long  retrievals. 

This  would  require  a combination  of  online  and  batch.  Thus,  retrieval 
specification  would  cake  place  online  and  the  actual  procasslng  wuld  be 
performed  in  batch.  There  might  also  be  the  need  for  some  standard  re- 
trievals which  would  be  run  quite  often  and,  thus,  would  be  predefined. 
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It  is  anticipated  that  the  repository  would  produce  reports  summariz- 
ing the  data.  Therefore,  this  subsystem  would  have  to  have  the  capabili- 
ties for  multi-attribute  qualification,  running  several  different  user 
programs  against  the  database,  specifying  output  columns  and  headings, 
placing  several  different  values  under  each  heading,  and  placing  limited 
free-form  remarks  under  certain  headings. 

There  is  the  need  for  the  generation  of  ad  hoc  reports  which  may  occur 
as  a result  of  an  online  batch  retrieval  or  may  originate  as  a batch  job. 

4.5.4  User  Programs 

In  order  to  support  the  repository  adequately,  the  system  will  have 
to  provide  for  the  running  of  a wide  variety  of  user  programs.  These 
programs  might  be  written  in  FORTRAN,  COBOL,  JOVIAL,  PL-1  or  some  other 
standard  programming  language,  and  require  the  use  of  a data  manipulation 
language  for  reading,  writing,  searching,  and  updating  the  database.  De- 
pending upon  the  function  of  the  program,  it  must  be  able  to  operate  on- 
line, in  batch  and/or  in  the  online/batch  mode. 

The  user  programs  will  vary  from  major  modeling  systems  to  statistical 
analysis  routines  acting  on  subsets  of  the  database.  Some  examples  of 
the  programs  that  would  be  required  for  statistical  analysis  include  mean, 
range,  variance,  standard  deviation,  confidence  intervals,  F-test,  T-test, 
chi-square  test,  analysis  of  variance,  regression  analysis,  analysis  of 
covariance  and  probability  distribution  fitting.  Programs  of  nonstatis- 
tlcal  nature  would  include  sort-merge,  qualification  to  obtain  a subset 
of  the  database,  and  various  routines  that  would  be  required  for  the  gen- 
eration of  reports. 

4.5.5  Database  Characterization  and  Structure 

A database  management  system  must  have  the  capability  to  define,  load 
and  update  the  data,  retrieve  on  the  attributes  of  the  data,  produce  re- 
ports and  be  able  to  interface  with  user  programs.  There  must  be  a data 
description  language  to  define  the  data  structure,  attribute  type  and 
length,  and  to  handle  coded  values  within  the  value  sets. 

As  discussed  in  Section  2,  there  are  three  SDR  databases: 

. Document  Description  Database 

. Historical  Database 

. Complementary  Database 

The  Document  Description  Database  contains  descriptive  information  on 
the  documents  within  the  document  collection.  This  database  is  made  up 
of  two  entitles,  seven  groups,  and  17  attributes  (Figure  4.6). 

As  defined  before,  an  entity  represents  an  object  or  event  concerning 
Che  software  development  process.  An  attribute  describes  the  entity  and 
is  Che  smallest  named  logical  unit  of  data  within  Che  database.  A group 
is  a named  collection  of  attributes. 
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FiRiire  4.6  Document  Description  Database 


The  Historical  Database  contiiiO'j  sunoarized  environment,  resource  uti- 
lization, production,  testing,  aad  change  data.  This  database  consists 
of  six  entities,  33  groups,  107  attributes  and  33  value  sets  (Figure  4.7). 
A value  set  is  the  set  of  all  possible  values  for  an  attribute.  The  ini- 
tial values  for  24  value  sets  are  illustrated  in  Figure  4.8.  The  values 
for  the  additional  nine  value  sets  will  be  determined  as  data  is  accepted 
into  the  database. 

The  Complementary  Database  contains  detailed  testing  and  change  data. 
This  database  consists  of  three  entities,  14  groups,  67  attributes  and 
20  value  sets  (Figure  4.9). 

Illustrated  in  Figure  4.10  are  the  entities  and  groups  of  the  three 
databases  shown  in  a hierarchical  manner. 

4. 5. 5.1  Data  Independence 

As  we  learn  more  about  the  software  development  process  and  the 
factors  that  affect  cost,  productivity  and  quality,  the  types  of  data 
that  are  collected  and  included  in  the  databases  will  change.  As  this 
will  be  an  evolving  process  rather  than  a sudden  change,  there  is  a 
need  to  reflect  these  modifications  minimizing  the  effects  on  the  op- 
eration of  existing  programs.  There  is  a need  to  provide  for  indepen- 
dence from  the  physical  storage  device,  the  ability  to  add  new  fields 
to  the  records,  and  the  ability  to  alter  relationships.  One  method 
of  minimizing  the  effects  of  these  changes  is  through  the  concept  of 
schema  and  subschema,  and  of  execution  binding. 

The  schema  is  a description  of  the  database  (Figure  4.11).  The  sub- 
schema is  a description  of  the  data  chat  is  needed  to  meet  Che  require- 
ments of  a particular  program.  The  schema  is  translated  by  a database 
management  system  into  a set  of  scored  schema  cables.  This  is  done 
separately  from  any  application  program.  Ac  program  compilation  time, 
Che  subschema  is  also  translated  into  a set  of  tables.  Then  at  execu- 
tion time,  Che  database  management  system  "binds"  these  two  cables 
together  in  order  to  provide  Che  data  as  described  in  the  subschema, 

Co  Che  program.  The  conventional  approach  is  to  "bind"  the  data  to 
the  program  at  compilation  time  rather  Chan  execution  time. 
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5.  System  Considerations 


5.1  Introduction 

This  section  contains  discussions  on: 

1.  systems  that  automatically  and  manually  collect  software  data, 

2.  data  entry  hardware  devices  chat  are  available  on  the  market  for 
consideration  as  tools  to  transform  hard  copy  data  to  computer- 
readable  form, 

3.  alternative  information  system  and  database  management  system  ap- 
proaches, and 

4.  alternative  approaches  to  document  library  automation 

The  reader  who  is  only  interested  in  the  recommendations  for  the  pilot 
facility  would  not  lose  the  flavor  of  the  repository  if  this  section  was 
skipped.  It  is  expected  chat  this  section  will  be  used  more  for  reference 
purposes. 

5.2  Data  Collection  Systems 

The  purpose  of  this  subsection  is  to  provide  a general  description  and 
data  applicability  to  the  repository  of  seven  representative  systems  that 
collect  software  experience  data.  The  purpose  of  these  systems  vary  and  are 
described  for  each  case.  Table  5.1  contains  an  indication  of  Che  types  of 
data  each  system  provides. 

5.2.1  Management  Data  Collection  and  Reporting  System 

The  Management  Data  Collection  and  Reporting  System  (MDC&R)  is  being 
developed  by  Che  IBM  Federal  Systems  Division  under  contract  to  RADC.  It 
was  initially  specified  in  the  Structured  Programming  Series  Reports  and 
is  part  of  the  Program  Support  Library  (PSL).0^3,  079,  087  purposes 

of  this  collection  system  are  Co  provide  management  with  information  for 
project  control  and  to  improve  our  understanding  of  the  software  develop- 
ment process.  They  have  identified  112  data  items  for  manual  and  auto- 
matic collection. 

They  divide  Che  data  into  two  major  classes,  plan  data  and  actual  data. 
The  plan  data  describes  the  characteristics  of  the  project  as  originally 
planned,  or  as  better  estimates  become  available;  the  actual  data  describes 
the  characteristics  of  the  project  as  the  development  progresses.  Each  of 
these  two  major  classes  is  further  divided  into  five  types:  project  envi- 
ronment, module,  computer  utilization,  resource  cost  and  program  production. 

Before  explaining  these  five  types,  it  Is  important  to  delineate  their 
five  levels  for  collecting  data.  The  unit  level  is  the  lowest  level  for 
collecting  data.  The  unit  is  a named  subdivision  of  a program  which  can 
be  treated  as  s single  entity.  The  second  level  is  the  program.  This 
is  the  lowest  level  that  can  be  assembled  (or  compiled)  and  executed  as 
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Tabic  3.1 

Date  Collection  Syctems 


En City /Croup  Naae 


Syctea  Name 


MDCiR  SIMOK  PET 


Project /CoBponent  Deacrlptlone 
Project 
Syetam 
Module 
Coaponent  ID 
PeraoT) 

Computer 

Environaent 

Peraon 

Computer 

General 

Technology 

Technology 

Resource  Utilization 
Calendar  Tlae 
Peraon  Time 
Person  Cost 
Computer  Time 
Computer  Cost 
Travel 
Travel  Cost 
Hisc.  Cost 
Total  Cost 

Production 

Documentation 

Instructions 

Change 

Test 

Error 

Corrections 

Software  Characteristics 
Quality 
Complexity 
Testing 
Constituents 

Test  Results  (Detailed) 

Test  Information 
Error  Information 
Correction 

Engineering  Change 
Change 


Key:  3 > Provides  the  majority  of  the  data  required 
2 • Provides  an  adequate  amount  of  the  data 
1 • Provides  s minimal  amount  of  data 
Blank  > No  data  is  provided 
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a single  entity.  The  third  level  Is  the  cc^uter  Job,  where  a Job  Is  made 
|p  up  of  one  or  more  Job  steps.  A Job  step  can  be  a compilation,  utility  run, 

etc.  The  fourth  level  Is  the  subsystem,  which  Is  comprised  of  one  or  more 
programs,  and  represents  the  computer  software  that  Is  developed  for  a 
project. 

The  project  environment  data  provides  a general  description  of  the 
project  and  of  the  software  system  to  be  developed;  and  characteristics 
of  the  personnel  and  the  customer.  The  module  data  describes  the  unit  and 
the  program  In  terms  of  dates,  programming  language,  and  name  of  program- 
mer. The  source  code  also  constitutes  part  of  the  module  data. 

The  computer  utilization  data  describes  the  use  of  the  computer,  by 
Job,  In  terms  of  computer  turn  around  time.  The  resource  cost  data  de- 
scribes the  costs,  by  subsystem,  for  materials,  personnel,  travel  and 
computer . 

The  program  production  data  describes  the  development  of  the  programs 
and  units  in  terms  of  number  of  compilations  and  changes;  number  of  up- 
date executions;  and  the  number  and  types  of  improvements  made  or  errors 
corrected . 

As  illustrated  In  Table  5.1,  the  Management  Data  Collection  and  Report- 
ing System  provides  a good  portion  of  the  kinds  of  data  required  for  the 
SDR  databases.  It  is  lacking  In  Its  collection  of  detailed  testing,  er- 
ror, and  correction  information  but  does  provide  some  summarized  testing 
and  error  data.  Data  on  component  descriptions,  constraints  and  priori- 
ties, quality,  and  engineering  changes  are  not  available  through  the 
system  but  could  be  provided  as  supplemental  information. 

5.2.2  Software  Implementation  Monitor 

The  Software  Implementation  Monitor  (SIMON)  is  being  developed  by 
Mitre  Corporation  under  contract  to  RADC.®2^”®23,  031,  085  xhe  major  pur- 
poses of  this  system  are  to  provide  for  the  systematic  and  consistent  col- 
lection of  data  for  research  into  factors  affecting  software  quality  and 
cost,  and  to  provide  technical  and  managerial  visibility  of  the  software 
development  process. 

They  have  defined  111  data  items  that  should  be  collected  during  the 
development  cycle.  The  data  la  divided  Into  five  major  types:  project, 
person,  subsystem,  module,  and  error-dlsc-lnformation. 

The  project  data  describes  the  project  as  a whole;  the  person  data  de- 
scribes the  assignments  of  programmers  to  subsystems;  the  subsystem  data 
describes  the  status  of  each  subsystem  in  terms  of  resources  used  (persons, 
computer,  etc.)  and  phases  complete;  the  module  data  describes  the  program 
or  unit  of  compilation  In  terms  of  complexity  measures,  program  attributes, 
and  scatus:  and  the  error-disc-lnformatlon  describes  the  error  -history  dur- 
ing development  including  number  and  types  of  discrepancies  and  errors  by 
subsystem  and  total. 
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SIMON  offers  a viable  approach  to  provide  data  to  the  SDR  but  It  lacks 
In  its  collection  of  detailed  testing,  error  and  correction  Information 
although  some  summarised  error  data  Is  provided.  Data  on  component  de- 
scriptions, personnel  experience,  constraints  and  priorities,  technology 
and  production  are  not  available  and  only  computer  and  total  cost  data 
Is  provided.  All  of  the  above,  except  for  production  data,  could  be  pro- 
vided as  supplemental  information. 

5.2.3  Program  Evaluation  and  Tester  System 

The  Program  Evaluation  and  Tester  (PET)  System  is  a test  evaluation 
tool  developed  by  McDonnell  Douglas  which  automatically  generates  self- 
metric  or  self-measuring  software  from  existing  software.®®^*®®’  Self- 
measuring  refers  to  the  ability  to  measure  the  thoroughness  of  testing  to 
which  a piece  of  software  has  been  subjected  by  analyzing  sensors  that 
have  been  Inserted  into  the  software  at  selected  points.  Summary  and  de- 
tailed reports  are  produced  that  quantify  the  degree  of  testing  to  which 
a program  has  been  subjected. 

The  types  of  dsta  collected  relate  to  the  paths  in  a program  that  have 
or  have  not  been  tested.  Although  this  system  does  not  collect  any  cost 
or  productivity  data,  it  could  be  used  In  conjunction  with  other  data 
collection  systems  to  provide  Information  on  software  validity. 

5.2.4  Automated  Software  Evaluation  System 

The  Automated  Software  Evaluation  System  (ASES)^^^  was  originally 
designed  by  the  University  of  California  at  Berkeley  under  the  sponsorship 
of  the  U.S.  Army  Safeguard  System  Evaluation  Agency  to  assist  in  the  de- 
velopment of  large  scale  software.  The  system  produces  reports  that  pro- 
vide a list  of  statements,  cross  reference  listings,  language  constructs, 
warning  messages  and  structural  characteristics  of  the  program.  It  pro- 
vides a static  analysis  by  producing  lists  of  undefined  labels,  unrefer- 
enced labels,  unreachable  statements,  statements  with  no  successors  with- 
in the  program,  and  direct  predecessors  to  all  labelled  statements.  It 
provides  for  error  detection  related  to  semantics,  language  constructs 
and  program  structure. 

The  software  data  collected  Is  similar  to  that  generated  by  a compiler; 
therefore,  the  data  collection  capability  is  very  limited.  This  limita- 
tion, together  with  the  fact  that  the  system  was  designed  to  operate  for 
a specific  programming  system  (CEKTRAK)  make  It  undesirable  as  a data 
collection  tool  for  the  SDR. 

5.2.5  The  Integrated  Mangement,  Project  Analysis  and  Control  Technique 

The  Integrated  Management,  Project  Analysis  and  Control  Technique 
(IMPACT)  is  a component  of  the  Software  Factory  developed  by  System  De- 
velopment Corpora ti on. The  purpose  of  IMPACT  is  to  assist 
the  manager  in  evaluating  project  progress  and  spot  potential  difficul- 
ties and  development  trends. 
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IMPACT  provides  for  the  storage  of  schedule,  resource  ellocetion,  per- 
sonnel assignment,  programs,  requirements  end  milestone  information. 

The  data  files  for  IMPACT  are  created  and  maintained  by  utilising  data 
coded  onto  forma  called  Task  Milestone  Worksheets.  These  worksheets  com- 
prise a variety  of  forms  all  of  which  are  required  to  compl-^tely  describe 
a project. 

The  first  of  the  series  of  worksheets  is  called  the  IMPACT  Information 
Item  Definition  Worksheet.  This  worksheet  is  used  to  record  program 
module  descriptions,  milestone  events,  documents  to  be  referenced  or 
written,  modifications  to  be  processed  and  validation  criteria  for  testing 
and  specification  reviews.  The  Module  Task  List  Worksheet  is  used  to 
record  schedules,  work  units,  resource  allocation  and  responsibility  as- 
signment data.  Also  entered  on  this  form  is  cross-reference  Information 
pointing  to  software  documentation,  milestones,  module  files,  design 
change  records  and  software  tests  and  validation  criteria. 

The  third  Task  Milestone  Worksheet  is  called  the  Module  Environment 
List.  This  form  is  used  to  record  module,  data  and  equipment  interdepen- 
dencies. This  data  is  used  to  establish  the  hierarchlal  structure  of  the 
software  system.  The  Task  Configuration  Worksheet  is  used  to  record 
tasks,  subtasks,  task  interdependencies  and  formulas  for  estimating  re- 
source requirements  per  unit  of  work  for  the  task.  The  Milestone  Event 
and  Software  Certification  Authentication  Log  la  used  to  record  mile- 
stone events,  software  test  schedules,  milestone  completion  dates  and 
Che  name  of  the  person  who  certified  the  milestone  hs  having  passed. 

This  information  is  used  to  report  software  configuration  status.  The 
Modification  Milestone  Log  is  the  sixth  Task  Milestone  Worksheet.  This 
form  is  used  to  record  the  processing  steps  to  be  followed  for  proposing, 
evaluation,  approving,  and  implementing  changes. 

The  last  Task  Milestone  Worksheet  is  used  to  record  personnel  assign- 
ments. This  data  includes  the  employee  name,  person  number,  skill  cate- 
gory, organization  code,  and  for  each  work  assignment,  the  module  name, 
cask  name  and  charge  number.  The  time  expended  on  each  cask  is  added 
to  Che  information  when  a cask  is  completed. 

In  addition  to  the  information  mentioned  above  as  manual  input  to  the 
IMPACT  data  base,  several  production  data  items  are  captured  automatically 
from  compiler  generated  information.  These  data  items  Include: 

1.  Nusiber  of  compilations  per  module 

2.  Number  of  tests  executed 

3.  Computer  time  expendid  on  coispilations  or  test 

4.  Module  size  (number  of  atacefflencs) 

5.  Cross  reference  information 

6.  Number  of  errors  encountered 

The  combination  of  the  manual  and  automatically  collected  data  constitu- 
cas  the  information  retained  in  the  IMPACT  database.  The  IMPACT  database 
comptisas  the  major  portion  of  the  project  control  data  and  is  the  major 
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source  of  infomstlon  used  by  software  developsent  managers  to  monitor 
project  activity. 

JHPACT  provides  a large  percentage  of  data  required  for  the  SDR  data- 
bases. It  Is  lacking  the  project  and  system  descriptions  including  con- 
straints, priorities  and  technologies  used.  However,  these  could  be  col- 
lected independently  of  the  IMPACT  System.  In  addition,  more  detailed 
configuration  management  data  is  collected  than  is  required  of  the  reposi- 
tory and  therefore  summarization  is  needed. 

5.2.6  RXVF  Automated  Verification  System 

The  RXVP  System  was  developed  by  General  Research  Corporation  for  fur- 
ther research  aimed  toward  identifying  and  solving  problems  of  software 
reliability  and  quality  and  evaluating  automated  tools  which  enhance 
software  reliability. 0^0-052,  058  RXVP  collects  static  and  dynamic  pro- 
gram behavior  data  by  the  insertion  of  software  "probes"  or  "sensors" 
at  each  decision  point  in  the  Fortran  program  so  the  outcome  of  each  de- 
cision can  be  recorded. 


All  of  the  data  collected  by  RXVP  is  collected  automatically.  The  data 
retained  in  the  database  is  related  to  test  results  and  module  behavior 
during  testing.  The  types  of  module  data  captured  Include: 

. name,  number,  type 

. number  of  statements,  symbols,  entries,  paths,  external  modules 

referenced,  paths  (2  types),  executable  statement,  branch  executions 
. complexity  measurements,  symbol  usage  checks 

. essential  paths  whose  execution  is  necessary  to  ensure  complete  test- 
ing coverage. 

The  RXVP  data  could  form  the  basis  of  an  entity  within  the  Complemen- 
tary Database  for  the  calculation  of  complexity  and  quality  measurements. 
However,  because  it  is  Fortran  limited  and  there  is  no  cost,  productivity 
or  project  description  data,  this  would  be  of  limited  utility  unless 
these  kinds  of  data  had  already  been  provided  through  other  data  collec- 
tion systems. 

3.2.7  Jovial  Automated  Verification  System 

The  Jovial  Automated  Verification  System  (JAVS)  was  developed  by  Gen- 
eral Research  Corporation  under  contract  to  RADC  and  is  a similar  system 
to  RXVP  operating  on  JOVIAL  source  code.®^^*  017  javS  analyzes  the 
JOVIAL  source  code,  assists  the  user  in  preparing  test  cases,  and  analyzes 
the  results  of  tests. 

As  with  the  RXVP  data,  the  JAVS  data  could  be  used  in  conjunction  with 
supplemental  cost  and  productivity  data  to  perform  studies  on  complexity 
end  quality. 
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5.3  Data  Entry  Hardware  Optlona 

The  data  entry  function  is  defined  as  those  operations  necessary  to  trans- 
form required  information  from  its  original  form  to  a form  most  suitable  for 
computer  processing.  This  subsection  presents  a discussion  of  alternative 
hardware  devices  that  assist  in  this  transformation  process. 

The  traditional  mode  for  achieving  the  data  entry  function  involves  the 
presentation  of  the  source  data  on  an  input  document  which  is  then  read  by 
an  appropriately  trained  and  equipped  operator  and  transformed  to  computer- 
readable  form  (such  as  punched  cards  or  tape,  or  magnetic  tape  or  disk,  etc.) 
by  the  entering  of  the  information  via  some  type  of  keyboard.  The  keyboard, 
device  itself  may  be  used  as  a stand-alone  unit  (i.e.,  keyboard,  control  and 
output  are  all  self-contained) , may  be  online  to  a host  computer  for  inter- 
active entry/response,  or  may  be  one  of  a multi-station  installation  (i.e., 
more  than  one  keystation  under  the  control  of  a single  processor  and  cen- 
tralized output  storage). 

5.3.1  Keyboard  Devices 

The  keypunch  has  been  around  for  many  years  and  will  continue  to  play 
a significant  role  in  the  future  in  data  entry  procedures.  However,  new 
technological  advances  of  the  past  few  years  allowing  the  relatively  in- 
expensive replacement  of  hard-wired  controller  logic  with  state-of-the- 
art  microprocessors  and  semi-conductor  memories,  coupled  with  cost  reduc- 
tions which  have  been  effected  for  magnetic  storage  drives,  now  allow 
a number  of  other  keyboard-to-storage  approaches  to  be  seriously  considered 
as  very  viable  alternatives. 

For  the  relatively  low  volume  card-oriented  environment  a buffered  key- 
punch should  be  considered.  At  approximately  $170  per  month,  little  more 
than  the  combined  cost  of  an  unbuffered  keypunch  and  verifier,  the  capa- 
bilities of  both  can  be  combined  into  one  unit,  with  the  added  benefit  of 
significantly  faster  operation  and  normally  many  more  allowable  program 
formats. 

Another  alternative,  occasionally  Justified,  is  the  keyboard  to  punched 
tape  device  typified  by  the  Teletype  ASR  33,  where  data  is  entered  onto 
punched  tape  in  an  offline  mode  and  then  transmitted  over  communications 
networks  to  a central  computer. 

A variety  of  stand-alone  keyboard-to-magnetlc  storage  devices  may  be 
used  to  score  data  onto  compuCer-compatiblc  magnetic  Cape,  magnetic  cape 
cassette  or  cartridge,  or  fixed  or  removable  disk  or  diskettes.  For 
these  devices  Che  stored  date  is  conveyed  to  the  central  computer  by  the 
pooling  onto  a magnetic  tape  or  disk  and/or  transmitting  the  consolidated 
information  via  data  lines.  These  devices  offer  significant  advantage 
over  keypunch  approaches  in  that  record  sizes  are  no  longer  limited  to 
the  80-  (or  96-)  character  constraint  of  a punched  card,  upper-  and  lower 
case  alphabetics  can  be  input  to  the  system,  the  storage  medium  can  be 
reused,  and  throughput  la  dramatically  Increased. 
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Stand'Clong  kcy/sagnctic  tape  kcyatations  coat  on  the  order  of  $150  to 
$250  per  aonth.  However,  at  Icaat  one  station  oust  be  equipped  with  a 
pooling  and/or  data  coea&unicationa  capability  which  will  add  another 
$250  to  $300  per  aonth. 

Because  of  their  added  reliability  and  alapliclty  of  use,  the  key  to 
cassette,  cartridge,  diskette,  and  disk  units  are  currently  in  auch  high- 
er demand  than  the  key /magnetic  cape  devices.  Key  to  disk  devices  start 
at  about  $400  per  month  and  the  others  at  about  $150  to  $250  per  month. 

As  with  Che  key  Co  magnetic  cape  devices,  however,  at  least  one  unit  must 
have  a pooling  and/or  data  coonDunicaclons  capability  which  may  add  another 
$200  or  $300  per  month. 

Keyboard  entry  can  also  be  effected  through  online  interaction  with  a 
host  computer.  If  this  approach  is  used,  online  editing  of  the  data  as 
it  is  entered  can  be  performed.  However,  there  is  increased  operator- 
associated  overhead  resulting  from  timesharing  response  delays  coupled 
with  the  added  communications  line  costs  if  the  central  computer  is  not 
onsite. 

If  online  daca  entry  is  elected,  either  hard-  or  soft-copy  (e.g.,  CRT 
display)  terminals  can  be  employed  for  the  data  entry  activities.  Hard- 
copy terminals,  in  general,  cost  less  than  their  soft-copy  counterparts 
and  offer  the  distinct  advantage  of  a hard-copy  record  of  all  transactions. 
The  soft-copy  units,  on  the  other  hand,  are  generally  more  reliable  and 
quiet,  and  besides  not  everyone  needs  a hard-copy  record  of  all  transac- 
tions from  the  terminal.  Costs  for  hard-copy  devices  can  range  from  under 
$50  pel  month  for  the  10  character  per  second  Teletype  KSR  33  to  about 
$150  per  month  for  a typical  30  character  per  second  device.  For  about 
$175  to  $200  per  month  plus  a modem,  a 120  cps  device  may  be  acquired. 

Soft-copy  units  are  primarily  CRT  displays  but  could  also  incorporate 
ether  techniques  such  as  LEDs  or  plasma  displays,  or  even  an  audible  sig- 
nal. Typical  limited  capability  display  devices  can  be  acquired  for 
about  $100  per  month  while  those  with  more  desirable  features  are  general- 
ly available  for  about  $150  per  month. 

In  many  cases,  the  same  hard-  and  soft-copy  devices  available  as  online 
data  terminals  are  also  available  as  integral  parts  of  the  keystations 
used  in  many  key/storage  devices. 

For  installations  with  substantial  data  entry  volumes,  multi-station 
key  to  storage  devices  should  be  given  serious  consideration.  Such  units 
are  considerably  more  costly  chan  stand-alone  devices  and  require  signi- 
ficantly more  planning  and  preparation  for  their  proper  use.  However, 
under  the  proper  circumstances,  the  vastly  enhanced  operator  productivity 
and  system  capabilities  in  the  form  of  relaxed  program  format  and  block 
site  restrictions  and  improved  editing,  verification,  and  validation 
capabilities,  can  make  it  all  more  than  worthwhile. 
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Being  a complete  computer  processing  facility  in  miniature,  multi- 
station key-entry  systems  come  at  a premium  price,  with  typical  costs 
running  from  about  $1,000  per  month  to  an  eight-station  installation  vith 
minimal  capabilities  to  about  $4,000  to  $5,000  per  month  for  more  elabor- 
ate capabilities  and  additional  stations.  The  cost  per  station  for  the 
average  eight  key  installation  generally  runs  between  $130  and  $200. 

5.3.2  Nonkeyboard  Data  Entry  Procedures 

In  many  Instances,  particularly  where  there  is  a high  volume  of  data 
and  there  are  trained  operators  in  a controlled  environment,  the  elimina- 
tion of  the  keyboard  entry  step  through  the  use  of  mark  or  character  re- 
cording can  be  most  effective.  For  such  instances  where  the  amount  of 
information  recorded  on  a single  document  is  small,  the  use  of  mark  re- 
cording may  be  very  much  in  order.  If  the  information  requirements  are 
such  that  the  data  entry  document  can  be  designed  to  be  accommodated 
through  the  use  of  pencil  marks  in  selected  areas  of  the  document,  this 
technique  could  Indeed  by  the  best  alternative. 

If  the  mark  sense  form  can  be  incorporated  onto  a punched  card,  costs 
for  the  required  mark  scanner  are  relatively  low  - typically  running  at 
about  $250  per  month.  Because  of  the  added  complexity  of  the  document 
feeding  mechanism,  scanners  for  other  sizes  of  marked  documents  are  some- 
what more  expensive,  typically  running  in  excess  of  $1,000  per  month. 

Such  mark  readers  can  be  used  offline,  recording  onto  magnetic  storage, 
usually  tape,  or  else  directly  online  to  a remote  Job  entry  (RJE)  terminal 
or  a host  computer. 

Where  remotely-generated  Information  must  be  processed  in  very  high 
volume,  the  use  of  machine  character  recognition  techniques  may  be  in  or- 
der. While  devices  do  exist  for  directly  processing  handwritten  charac- 
ters, their  high  cost  coupled  with  the  prevailing  high  rejection  rates 
for  such  documents  except  for  very  strictly  constrained  circumstances, 
precludes  their  serious  consideration  at  this  time. 

On  the  other  hand,  many  successful  applications  have  been  affected 
with  the  use  of  automatic  character  recognition  techniques  using  highly 
stylized  character  fonts.  A number  of  such  fonts  have  been  standardized 
in  the  Industry  and  there  arc  a variety  of  page  and  document  readers 
available  to  directly  generate  the  required  digitized  information  from 
such  documents.  Typical  optical  character  scanners  for  the  stylized  fonts 
can  be  installed  for  upwards  from  $1,500  per  month  while  those  recognizing 
limited  handprinted  information  usually  cost  In  excess  of  about  $2,500 
per  month. 

Another  viable  option  for  large  volume  Installations  Is  the  use  of  a 
so-called  mixed  media  system  which  combines  an  optical  character  recogni- 
tion (OCR)  or  optical  mark  recognition  (OMR)  reader  with  a multi-station 
key  to  storage  system.  This  approach  allows  source  data  to  originate 
from  both  traditional  Input  forms  and  from  scannable  forms  as  is  most 
appropriate.  It  also  affords  Improved  error  correction  facilities  for 
resolving  unrecognized  character  probleu  with  OCR  processing. 


I 

I 
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5.3.3  Futurt  Trends 


It  has  bacn  astisatad*  that  at  the  and  of  1974,  the  total  U.S.  instal- 
lad  bast  of  data  antry/conaunications  facllitias  a&ountad  to  about  640,500 
units,  about  half  of  which  ware  buffarad  and  unbuffarad  kaypunching  de- 
vicas.  Bowavar,  the  aaaa  aourca  which  pradicts  a 15  parcant  annual  growth 
In  kayboard  basad  installations  through  1979,  pradicts  the  following 
growth  factors  by  1979  as  a aultiple  of  1974  installations: 

. Multi-station  keyboard  to  storage  devices  — Up  250  tiaas 

. Intelligent  tarvinals  — Up  7.5  tiaas 

. Soft-copy  editing  and  conversational  tar- 

ainals  — Up  3.4  tiaas 

. Hard-copy  editing  and  conversational  ter- 
minals — Up  1.2  tiaas 

. Stand-alone  keytodisk  storage  devices  — Up  1.3  times 

. Keypunches  — Down  1.4  tiaes 

. Stand-alone  key  to  tape  devices  — Down  7.7  times 

There  are  presently  soae  3000  to  4000  OCR  units  and  an  unknown  number 
of  OMR  units  installed,  OCR  has  not  grown  to  the  extent  that  was  predicted 
in  the  1960s  when  the  first  feasible  system  was  developed.  Unless  proce- 
dures in  the  area  of  error  recognition  are  vastly  improved  and/or  more 
effective  handwritten  character  readers  are  evolved,  growth  in  the  use  of 
this  approach  will  probably  continue  to  be  stunted. 

OMR  techniques  are  very  effective  in  highly  constrained  and  specialized 
situations.  Modest  growth  in  their  use  is  probably  to  be  expected. 

As  can  be  .«een  by  the  chart  above,  however,  the  most  spectacular  growth 
is  expected  to  take  place  in  the  use  of  multi-station  key/storage  devices, 
with  250  keystations  in  use  by  1979  for  every  one  that  was  around  in  1974. 

Use  of  soft-copy  CRT  display  terminals,  presumably  for  real  time  appli- 
cation, will  more  than  triple  while  hard-copy  conversational  terminals 
and  stand-alone  keyboard  to  disk  and  diskette  units  will  show  only  modest 
gains  through  1979. 

i As  more  installations  switch  to  the  newer  techniques  for  key-entry  the 

actual  keypunch  population  will  decline  about  30  percent  by  1979. 

By  far  the  biggest  losers,  however,  are  expected  to  be  the  stand-alone 
key  to  tape  devices,  with  a decline  of  about  67  percent  in  installed  key- 
( stations  for  the  five-year  period  from  1974.  As  indicated  earlier  in 

I this  report,  actual  sales  of  key  to  compatible  tape  devices  have  been 

1 steadily  declining  since  the  early  1970s  in  favor  of  the  easier  to  use 

I alternatives  offered  by  the  key  to  cassette  or  cartridge  or  key  to  disk 


*EDP  Industry  Report,  published  by  International  Data  Corp.,  Valthaa,  Hass., 
1975. 
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or  diskette  approaches  in  recent  years  will  most  certainly  have  a major 
impact  on  the  key/cassette  and  key/cartrldge  market.  After  all,  the 
diskettes  are  a thousand  times  faster  (in  search  mode)  chan  cassettes  or 
cartridges,  contain  about  the  same  amount  of  data  and  cost  roughly  Che 
same. 

These  figures  are  also  borne  out  by  a spring  1975  Datapro  survey  re- 
garding sice  intentions  to  change  data  entry  facilities.  Over  half  (52 
percent)  of  the  keypunch  facilities  and  key  to  cassette  facilities  (58 
percent)  and  two  thirds  of  Che  key  Co  compatible  Cape  facilities  (64  per- 
cent) plan  Co  replace  their  devices  in  the  immediate  future,  while  only 
about  18  percent  of  the  key  to  disk  and  key  to  diskette  facilities  con- 
template any  near-term  changes.  The  actual  intention  figures,  in  percent, 
are  shown  below: 


Users  of  Che 
Following 
Types 
of  Devices 

Plan 

CO  Change 

CO  the  Following  Types  of 

Devices 

Online 

Tape 

Key/ 

Diskette 

Multi-Station 

Key/Dlsk 

Other 

Keypunches 

17 

4 

12 

26 

2 

Key/Tape 

30 

- 

16 

18 

2 

Key/Cassecce 

0 

0 

42 

S 

8 

Key/Diskette 

11 

0 

— 

4 

4 

Miltl-Staclon 

Key/Disk 

13 

0 

1 

— 

4 

Another  notable  future  trend  is  the  further  blending  of  many  of  Che 
Individually  defined  data-encry  techniques  for  individual  applications. 

By  1979  Che  categories  of  data  entry  devices  as  defined  in  this  report 
will  probably  no  longer  hold.  The  advent  of  mixed-media  systems  and 
multi-station  systems  with  remote  data  entry  capabilities  were  breifly 
discussed  in  this  report.  To  this  must  be  added  the  evolving  use  of  such 
facilities  for  remote  batch  entry,  for  accommodating  sort  and  report  gen- 
eration, as  well  as  ocher  sophisticated  software  capabilities,  and  the 
ability  of  multi-station  CPUs  to  communicate  among  themselves  as  well  as 
with  Touchcone  telephones. 

5.3.4  User  Guidelines 

While  there  arc  obviously  no  hard  and  fast  rules  of  thumb  In  planning 
for  data  entry  needs.  In  view  of  Che  technological  discussions  contained 
In  the  earlier  sections  of  this  report,  and  the  future  Industry  expecta- 
tions as  discussed  in  the  preceding  subsection,  a number  of  comments  can 
be  made,  most  of  which  are  self-evident: 
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. At  wlcb  «ny  eoBputtt  tyttcBt  tottl  tytttB  cotct  for  the  date  entry 
ctptbllltiet  thould  be  taken  Into  account  — not  jutt  hardwere  device 
cog(g.  Such  coata  above  and  beyond  the  hardware  Include: 

- peraonnel  (operator  and  clerical  support) 

> training 

> acdla  handling  and  storage 

> office  apace  utilisation 

> data  convaralont 

> back-up  procedures 

- conputcr  preprocessing  tine 

- software  prograacing 

^ for  very  snail  volune  situations*  unbuffered  keypunches  should 

nomally  be  rejected  in  favor  of  other  alternatives. 

. Because  of  the  obsolete  nedia  and  technologies  Incorporated,  key  to 
punched  tape  and  key  to  conpatlble  nagnetic  tape  entry  devices  should 
not  nomally  be  considered. 

, Where  key  to  diskette  or  key  to  disk  pack  facilities  are  cost  justi- 
fied, they  should  always  be  given  greater  consideration  over  key  to 
cassette  devices  unless  the  application  is  limited  to  data  capture 
and  low  volume  editing. 

. In  general,  all  other  things  being  equal,  multi-station  key  to  storage 
data  entry  devices  or  optical  character  recognition  techniques  are 
best  suited  for  installations  with  high  data  entry  volumes,  while 
ether  stand-alone  key  to  storage  devices  are  more  appropriate  for 

lower  volume  installations. 

For  further  guidance,  the  reader  is  referred  to  the  matrix  in  Table 

5.2. 

5.4  Information  System  Alternatives 

There  are  three  basic  hardware  alternatives  for  fulfilling  the  general 
processing  requirements  for  the  software  data  repository.  They  are  the 
utilisation  of: 

1.  The  IIADC  Computer  Center 

2.  Off-site  facilities 

3.  A dedicated  mini-computer 


Belov  is  a discussion  of  each  of  these  alternatives  highlighting  the 
atabasc  management  system  availability.  It  is  assumed  that  the  alternatives 
eve  operational  system  software  including  an  operating  system,  high  order 
mnguage  compilers,  and  a set  of  utility  programs. 
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5.4.1  The  RADC  Computer  Center 


The  Information  Sciences  (IS)  Division  of  EtADC  Is  currently  maintaining 
two  Honeywell  6180  computer  systems  to  provide  services  for  various 
computational  needs  at  RAOC.  The  GCOS  and  Multlcs  Operating  Systems  are 
available. 

Seven  database  management  systems  have  been  examined  for  possible 
use  by  the  SDR.  The  system  characteristics  of  each  are  summarized  In 
Table  5.3  including  the  designation  of  the  appropriate  operating  system, 
the  organization  responsible  for  maintaining  the  system  at  RADC, 
and  an  indication  of  whether  its  strengths  are  in  the  area  of  self- 
contained  capabilities  or  host  language  capabilities.  Below  is  a list 
of  the  major  characteristics  for  each  system.  The  current  number  of 
users  is  classified  into  four  categories.  None  means  that  there  are  no 
current  users,  small  implies  less  than  10  users,  medium  implies  that 
there  are  between  10  and  50  users,  and  large  means  that  the  system  is  used 
by  over  50  users. 


For  a definition  of  terms  used  below,  see  Appendix  A. 
DM-l  Characteristics  ^^^>^20 


Data  Structure-Hierarchical  (partially  inverted) 

Host  language 

Self-contained 

Number  of  users  - None 

Special  Features  - Designed  to  provide  data  independence  features. 


JANUS 


Characteristics 


218,  219,  223 


. Data  structure  - Relational 
. No  host  language 
. Self-contained 
. Number  of  users  - Small 

. Special  Features  - Extensive  statistical  analysis  package. 


I-D-S 


Characteristics 


214, 


215 


. Data  Structure  - Network 
. Host  language 
. Self-contained  (limited) 

. Number  of  users  - Large  2ns 

. Special  Features  - I-D-S/II  consistent  with  the  CODASYL/DDCL  73. 


MDQS 


Characteristics 


212, 


213 


. Data  Structure  - Network,  Hierarchical,  Singular 
No  host  language 
. Self-contained 

Number  of  users  - Medium 

Special  Features  - Coonercial  version  of  the  World  Wide  Data  Man- 


5-15 


i 


Table  5.3 


Syatema  are  new  Mult  lea  releases  and  their  capability  has  not  yet  been  established 


agemenC  System  (WWDMS).  Interfaces  with  I-D-S  files. 
226 

FMIS  Characteristics 

. Data  structure  - Fully  inverted 
. No  host  language 
. Self-contained 
. Number  of  users  - Small 

. Special  features  - Efficient  retrieval  capabilities. 
MRDS  Characteristics 


Data  structure  - Hierarchical,  Network,  Relational 

Host  language 

No  self-contained 

Number  of  users  - Small  or  none 

Special  features  - A function  within  the  Multics  Data  Base 
Manager  (MDBM) . 


MIDS  Characteristics 


Data  structure  - Hierarchical,  Network 

Host  language 

No  self-contained 

Number  of  users  - Small  or  none 

Special  features  - A function  within  the  Multics  Data  Base 
Manager  (MDBM) . 


5.4.2  Off-Site  Facilities 

Services  can  be  purchased  from  commercial  data  centers,  from  installa- 
tions connected  to  the  ARPANET,  and  from  commercial  networks.  This  alterna- 
tive offers  a wide  variety  of  possibilities  through  the  many  different 
hardware/software  services  available. 


Attached  to  the  ARPANET  are  a number  of  major  large  scale  computers  in- 
cluding IBM,  CDC,  DEC  and  Honeywell  computers.  Ocher  network  services  of- 
fer a variety  of  processing  capabilities.  Some  of  Che  commercial  network 
possibilities  are: 

- INFONET 

Computer  Sciences  Corporation 

- DATANETUORK 

Honeywell  Information  System 

- BCS  NETWORK 

Boeing  Computer  Services 

- UNA 

Utility  Network  of  America 

- TYMNET 

Tymehare,  Inc. 
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The  database  aanagement  systao  alternatives  are  greatly  expanded  when 
considering  to  use  one  of  the  network  services.  Some  of  the  better  known 
DBMS  are  TOTAL,  ADABAS,  IMS,  SYSTEM  2000  AND  I0M5  (Table  5.4}  The 
basic  capabilities  are  listed  belov.230 


TOTAL  Characteristics 


. Data  Structure  - Network 
. Host  language 
. No  self-contained 
. Number  of  users  - Large 

ADABAS  Characteristics 


. Data  Structure  > Network  (partially  Inverted) 
. Host  language 
. Self-contained 
. Number  of  users  - Large 

IMS  Characteristics 

. Data  structure  - Hierarchical 
. Host  language 
. Self-contained 
. Number  of  users  - Large 

SYSTEM  2000  Characteristics 


. Data  structure  - Hierarchical  (partially  inverted) 

. Host  language 
. Self-contained 
. Number  of  users  - Large 

IDMS  Characteristics 

. Data  structure  - Hierarchical,  Network 
. Host  language 
. No  self-contained 
. Number  of  users  - Large 

The  cost  to  purchase  these  database  management  systems  vary  from 
$X,000  for  a basic  TOTAL  System  to  $120,000  for  ADABAS  with  full  capability. 
However,  If  these  systems  have  been  purchased  by  the  commercial  network 
vendors  themselves,  the  cost  is  distributed  depending  upon  use. 

The  cost  of  service  for  both  the  hardware  and  software  from  these  com- 
aarclal  networks  varies  with  usage.  The  major  cost  elements  are  for  on- 
line database  storage,  CPU  and  1/0  usage,  local  data  entry  and  remote 
batch  terminals,  and  communication  services.  A gross  estimate  for  a 
pilot  facility  based  on  previous  projects  offers  a cost  range  of  $8,000 
to  $19,000  per  month. 


I 

J 

i 
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An  •stlMtc  using  Boeing  Cosipucer  Services  end  the  IDMS  datebese  nan- 
•geaent  on  a surcharge  basis  was  developed.  The  assuaptlons  aade  were 
that  the  application  programs  would  occupy  less  than  lOOK  bytes  of 
primary  storage,  there  would  be  500  retrieval  and  update  transactions 
per  day,  approximately  60  million  characters  of  online  storage  would  be 
required,  and  an  estimated  thirty  (30)  minutes  per  day  of  CPU  time  would 
be  required  for  this  level  of  activity.  The  total  ninthly  cost  estimate 
was  $16,500  and  was  broken  down  as  follows: 


CPU  and  I/C  Service 

$ 6,000 

IDMS  Surcharge 

2,400 

One  3330  Spindle 

1,300 

Four  dedicated  ports 

2,000 

Low  speed  terminals  (6) 

550 

Low  speed  terminal  maintenance 

150 

RJE  terminal  and  supplies 

1,100 

Communication  service 

3,000 

5.4.3  A Dedicated  Mini>Computer 

The  third  alternative  is  the  use  of  a dedicated  aini-computer  to 
perform  the  information  processing  for  the  SDR.  Some  mini-computer 
systems  offer  a subset  of  database  management  capabilities  as  discussed 
previously.  These  systems  are  termed  file  management  systems  and  pro- 
vide the  capabilities  for  data  entry  and  query,  but  have  limited 
capabilities  for  database  structuring  and  maintenance. 

The  following  is  a list  of  mini-computer  systems  that  provide  data- 
base management  or  file  management  capabilities 

- Data  General  ECLIPSE 

- General  Automation  18/30 

- Honeywell  60/62 

- IBM  System  3 

- Hewlett  Packard  3000 

- Varian  75 

- DEC  PDP  11 

A cost  comparison  for  three  mini-computer  systems  is  provided  in 
Table  5.5.  The  hardware/software  configuration  for  each  is  listed 
below: 

Varian  75/Vortcx  Il/Total 

. 126K  words  parity  core  memory 

Hardware  floating  point 

. 16  line  data  communication  multiplexor 

93  million  byte  disk 
75  ips,  600  bpi  magnetic  tape  unit 

. 600  1pm  alactrostatic  printer /plotter 

Hultiuscr  operating  system 
TOTAL  Database  manegement  system 


HP  3000/MPE/IMACE 


. Model  HP  32402C 
. 126  K byte  perity  core  senory 

. Tvo  47  ■llllon  byte  disks 
. 15  Billion  byte  eyeten  disk 

. Scientific  end  CoBserciel  Fitware 
. 16  line  Bultiplexor 

. 500  Ipo  electrostatic  printer  plotter 

. 45  ips,  800  bpi  magnetic  tape  unit 

Multi-user  operating  systeo 
. IMAGE /QUERY  Database  manageaent  system 

PDP  11/70-lAS-IDMS 

. Model  11/70  HA 
. 128  K word  parity  core  nenory 

. 512  K word  swapping  disk 

. 66  Billion  byte  disk  unit 

. hardware  floating  point 
. 45  ips,  1600  bpi  tape  unit 

. 500  1pm  electre static  printer  plotter 

. 16  line  multiplexor 

. Multiuser  operating  system 
. IDMS  Database  management  system 

5.5  Document  Library  Implementation  Considerations 

The  previous  sections  assumed  that  the  description  of  the  documents 
would  be  stored  in  a computerized  database  (The  Document  Description 
Database).  However,  there  are  alternate  methods  for  handling  this  in- 
formation, their  effectiveness  dependent  upon  the  size  and  utilization 
of  the  Library. 

The  primary  need  of  the  software  document  library  is  one  common 
to  all  libraries  - the  need  for  an  efficient  storage  and  retrieval 
system. 

In  order  to  provide  optimal  satisfaction  of  users'  research  needs, 
any  library  requires: 

1.  A means  of  selecting  documents  containing  information  relevant 
to  the  purpose  of  its  users. 

2.  A peans  of  describing  (indexing)  such  documents  in  an  accurate 
consistent  manner  so  that  they  can  be  subsequently  retrieved 
upon  the  user's  request. 

3.  Convenient  storage  and  accessibility  to  both  the  documents  and 
the  indexes  describing  these  documents. 

4.  The  ability  to  manipulate  indexed  information  for  search  and 
retrieval. 


1 
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5.  Some  form  of  meaningful,  organized  output  to  describe  the  results 
of  a search. 

The  Defense  Documentation  Center  (DDC)  provides  a bibliographic 
service  to  eligible  users  through  their  Technical  Report  (TR)  Program. 
The  types  of  documents  Included  In  this  program  are  the  formally  docu- 
mented scientific  ar.d  technical  results  of  Defense-sponsored  research, 
development,  teat  and  evaluation. 

The  Technical  Report  Program  Is  a major  source  for  documents  related 
to  Defense-sponsored  projects  In  software  engineering.  To  supplement 
these  reports,  other  bibliographic  services  (e.g.  ACM  Computing  Reviews) 
should  be  used. 

The  DDC  reports  are  assigned  an  AD  (Accessioned  Document)  number 
and  are  categorized  Into  a two-level  arrangement  consisting  of  22 
major  subject  fields,  with  a further  subdivision  of  168  related 
subject  groups.  The  subject  groups  related  to  software  engineering 
can  be  further  subdivided  to  reflect  the  special  level  of  Indexing 
needed  for  the  repository. 

Certain  procedures  are  common  to  document  processing  within  any 
storage  and  retrieval  system  regardless  of  whether  It  Is  a manual, 
semi-automatic  or  computer-oriented  system.  These  procedures  Include 
the  evaluation,  description  (Indexing)  and  storage  of  documents.  Pro- 
cedures for  the  Input,  storage  and  manipulation  of  descriptive  In- 
formation, however,  vary  depending  upon  the  particular  system  being 
employed.  These  procedures  are  described  within  the  context  of  each 
system  as  It  Is  discussed. 

5.5.1  Manual  Methods 

Information  listed  on  the  Document  Description  Form  (Figure  3.2)  are 
transcribed  to  a series  of  5x8  Index  cards.  Several  Index  files  are 
established  using  this  Information.  The  primary  files  created  are: 

1.  a complete  bibliography  - In  the  form  of  an  abstract  reference 
card.  This  card  contains  all  the  Information  found  In  the  record 
and  Is  filed  numerically  by  accession  number. 

2.  personal  author  and  corporate  author  files  - these  cards  lists 
author,  title,  source  and  date  of  publication  and  document  acces- 
sion number.  These  cards  are  filed  alphabetically  according  to 
the  author's  surname  or  the  corporate  name. 

3.  a subject  file  - this  file  la  set  up  In  one  of  two  ways: 

a.  a keyword  Is  listed  as  subject  heading,  followed  by  the 

author,  title,  source  and  date  of  publication,  and  the  docu- 
ment accession  number.  If  s document  contains  Information 
covering  loore  than  one  subject,  a separate  card  is  prepared 
for  each  keyword  used.  These  cards  are  filed  alphabetically 
by  subject  heading. 
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b.  one  card  It  prepared  for  each  aubject  cero  in  the  Index.  On 
thla  card  la  Hated  only  the  accession  numbers  of  all  documents 
dealing  with  that  particular  subject.  These  cards  are  filed 
alphabetically  by  subject  terms. 

Manipulation  and  retrieval  of  indexed  information  becomes  a difficult 
and  inaccurate  process  as  the  site  of  the  file  and  the  number  of  search 
requests  increase.  A manual  system  becomes  inadequate  when  documents 
■ust  be  aearched  vlth  a combination  of  index  terms.  Each  Index  term  is 
searched  individually,  pulled  from  the  file  and  then  crossmatched  against 
the  other  search  tersks.  This  type  of  manipulation  increases  the  possib- 
ility for  human  error,  especially  in  missing  matches.  The  index  list 
and  bibliographic  information  is  then  typed  in  the  form  of  a search 
report. 

A mr.nual  system  requires  much  typing  and  duplication  of  Informacion. 

In  addition  to  at  least  four  different  index  cards  for  each  document,  all 
bibliographic  and  index  lists  would  have  to  be  typed  by  clerical  person- 
nel. These  files  are  periodically  updated  to  reflect  current  file  status. 
Again,  this  requires  considerable  amount  of  retyping  material  already 
found  in  the  file. 

A manual  system  is  recommended  only  when  the  document  collection  is 
comparatively  small,  and  the  search  requests  are  few  in  number  and  not 
exceedingly  specific. 

5.5.2  Semi-Automated  Methods 

The  semi-automated  system  differs  from  the  manual  system  in  its 
handling  of  the  subject  terms.  An  inverted-term  matrix  retrieval  system 
is  used  for  the  subject  terms. 

An  inverted-term  matrix  retrieval  system  involves  the  preparation  of 
a file  record  for  each  concept  of  term  to  be  indexed.  The  system  used 
by  the  Reliability  Analysis  Center  employs  an  6-1/2  by  11  inch  card 
(file  record)  for  recording  each  concept.  There  are  10,000  possible 
items  (documents)  to  which  accession  numbers  are  assigned.  The  prepara- 
tion of  one  such  concept  card  actually  amounts  to  performing  one  com- 
plete search  of  the  file.  A request  for  the  documents  pertaining  to  a 
particular  concept  (i.e.,  cost  estimation,  reliability,  modeling,  etc.) 
is  answered  by  listing  the  document  accession  number  for  which  holes  are 
drilled  on  the  particular  concept  card.  The  great  power  of  the  method, 
however,  lies  in  the  ability  to  combine  concepts  and  issnedlately  read  out 
the  accession  numbers  pertaining  to  such  combinations. 

The  advantages  of  this  system  are  versatility,  availability  of  conven- 
ient and  inexpensive  equipment,  and  savings  in  index  acanning  tine  re- 
sulting from  the  inverted  nature  of  the  index  record. 

The  most  important  feature  of  the  inverted-ters^matrix  indexing  sys- 
tem is  the  versatility  which  it  gives  the  indexer.  This  versatility 
arises  from  the  fact  that  the  indexer  need  not  foresee  all  the  possible 
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combination  of  concepts  which  will  be  of  Interest  to  the  Inquirers.  New 
concepts  can  be  added  to  the  Index  merely  by  adding  new  concept  records. 

Old  concepts  can  be  easily  modified  or  changed  by  eliminating  from  the 
file  only  those  concept  records  which  are  Involved.  The  system  can  even 
be  used  to  find  Items  which  do  not  Include  a specific  concept. 

The  disadvantages  of  this  system  are  some  of  the  same  as  those  of  the 
manual  system.  As  there  Is  still  a need  for,  at  a minimum,  a bibliography 
file  and  a personal  author  file,  there  Is  the  requirement  of  typing  dupli- 
cate Information. 

5.5.3  Computer-Readable  Methods 

The  Information  listed  on  the  Document  Description  Form  is  keystocked 
and  entered  Into  the  database.  Although  information  storage  and  retrieval 
requirements  do  not  require  the  full  power  of  a database  management  system, 
the  capabilities  of  defining  updating  and  querying  the  database  can  be 
effectively  utilized. 

The  output  capabilities  are  greatly  enhanced.  In  addition  to  the  gener- 
ation of  a bibliography  and  reports  on  the  results  of  literature  searches, 
the  following  special  reports  can  be  produced: 

- KWIC  (Key  Words  In  Context) 

In  this  report,  significant  words  In  each  title  are  permitted  and 
listed  alphabetically  so  that  each  term  serves  as  an  access  point 
to  the  particular  document  In  which  It  appears. 

- KWOC  (Key  Words  Out  of  Context) 

This  reports  lists  key  words  assigned  to  each  document  and  cross-re- 
ferences each  keyword  to  Its  document  title.  This  Index  Is  quite 
useful  In  determining  document  content,  since  titles  are  often  vague 
or  do  not  always  list  all  types  which  are  covered  In  the  document. 

The  advantages  of  a computer-readable  system  are: 

- The  ability  to  manipulate  any  number  of  Index  terms  at  one 
time,  thereby  providing  depth  and  specificity  which  enhances 
the  retrieval  capabilities  of  the  system 

- The  elimination  of  large  amounts  of  time  and  repetitive  cleri- 
cal work  Involved  In  the  generation  of  lists  and  creation  of 
file  cards 

- The  ability  to  provide  the  user  with  organized,  meaningful 
reports  which  describe  the  results  of  e literature  search 

- The  ability  to  generate  e bibliography  by  automated -means 

- The  ability  to  handle  large  volumes  of  information  In  a re- 
latively short  time 
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Tht  di»*dv«nt»gtt  of  • cooputer  readable  ayetea  are: 

- The  need  for  aora  highly  qualified  people  to  parfom 
the  task  of  data  entry  and  simple  retrievals 

• Cost  of  utilising  the  computer  system 

••  Dependency  on  the  availability  of  the  computer  eystem 
5.5.6  Distribution  Considerations 

An  Important  factor  to  consider  when  designing  a document 
library  is  the  level  of  document  dissemination.  The  library  can  be  a 
lending  library,  a distribution  center,  a reference  library,  or  a com- 
bination of  the  above.  Factors  to  consider  are  copyright  permissions, 
proprietary  and  security  needs,  and  the  costs  of  reproduction  and 
shipping. 
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6.  Software  Data  Repository  Design 
6.1  Introduction 

The  functional  model  as  presented  in  Figure  2.1  is  the  functional  mod- 
el of  all  the  stages  within  the  software  data  repository  life  cycle.  As  the 
repository  evolves,  it  is  expected  that  the  Information  content  of  the  data 
will  change,  the  data  collection,  input  processing,  report  production,  consult- 
ing services,  and  the  engineering  and  data  analysis  processes  will  expand,  and 
that  a greater  quantity  and  a better  quality  of  outputs  will  result. 

A subset  of  the  services  and  outputs  described  in  Section  2 shall  be 
provided  through  the  implementation  of  a pilot  facility.  This  pilot  facility 
should  be  developed  so  that  continous  services  be  provided  to  the  software 
development  community  as  the  pilot  facility  expands  into  a fully  operational 
center. 


This  section  presents  the  design  for  the  Software  Data  Repos- 
itory including  the  development  and  operation  of  a pilot  &cility  and  the  ex- 
pansion to  a fully  operational  center. 

6.2  Scope  of  the  Pilot  Facility 

It  is  recommended  that  the  Software  Data  Repository  pilot  facility  be 
established  as  a separate  contractor  operated  function  and  be  administered  and 
guided  by  the  Rome  Air  Development  Center  (RADC),  U.S.  Air  Focce.  The  facility 
should  be  located  in  Rome,  New  York  and  utilize  the  computer  systems  available 
at  RADC. 


6.2.1  Source  Data  and  Documents 

The  two  major  classes  of  information  that  form  the  inputs  to  the 
pilot  facility  shall  consist  of  production/development  data  and 
textual  information.  The  production/development  data  shall  provide 
experience  information  on  developing  and  maintaining  computer  software. 
The  major  sources  of  this  data  for  the  pilot  facility  include  datasets 
that  RADC  has  contracted  to  purchase  and  data  that  is  acquired  during 
the  development  and  operation  of  the  pilot  facility. 

The  five  known  productlon/development  dataset  sources  and  their 
characteristics  are  listed  in  Table  6.1.  The  first  three  sources  pro- 
vide detailed  testing  and  error  data  while  the  third  source  also  pro- 
vides some  environment  and  resource  utilization  data.  The  last  two 
sources  provide  summarized  testing  and  error  data  in  addition  to  en- 
vironment, resource  utilization,  and  production  date. 

The  second  dess  of  Information  for  the  pilot  facility  shell  con- 
sist of  textual  or  expository  information  directly  related  to  improv- 
ing Che  process  of  designing,  developing,  testing  end  maintaining  com- 
puter software.  These  documents  include  research  and  development  re- 
ports, technical  end  journal  ercicles,  handbooks,  conference  proceed- 
ings, end  specifications  end  standards. 
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The  major  sources  for  this  informacloa  are: 

A)  Ck)vernment  reports  relating  to  software  engineering 

- Unclassified,  unlimited  distribution 

National  Technical  Information  Service  (NTIS) 

- Classified  or  limited  distribution 

Defense  Documentation  Center  (DDC) 

NTIS  is  an  agency  of  the  U.S.  Department  of  Commerce  and  is  a cen- 
tral source  for  the  public  sale  of  Government-sponsored  research, 
development  and  engineering  reports  and  other  analyses  prepared  by 
Federal  agencies,  their  contractors  or  grantees. 

DDC  is  the  clearing  house  for  the  Defense  Department's  collection 
of  research  and  development  in  virtually  all  fields  of  science  and 
technology.  Research  and  development  activities  within  the  U.S.  Gov- 
ernment and  their  associated  contractors,  subcontractors,  and  grantees, 
with  current  Government  contracts,  are  eligible  to  receive  most  of  the 
Information  from  the  DoD  data  banks  located  at  DDC. 

B)  DoD  software  development  specifications  and  standards 

- Naval  Publications  and  Forms  Center 

All  documents  listed  in  the  DoD  Index  of  Specifications  and  Stan- 
dards can  be  ordered  from  the  Naval  Publications  and  Forms  Center. 

This  DoD  Index  includes  unclassified  Federal,  Military  and  Departmental 
specifications,  standards,  and  related  standardization  documents,  and 
those  Industry  documents  which  have  been  coordinated  for  DoD  use. 

C)  All  other  documents  related  to  software  engineering 

- Author  or  Publisher 

It  is  estimated  that  the  number  of  textual  documents  for  the  pilot 
facility  will  be  in  the  range  of  2,300  to  3,300. 

From  the  textual  and  production/development  Inputs,  the  Document 
Description  Database,  the  Historical  Database,  and  the  Complementary 
Databases  shall  be  generated. 

6.2.2  Document  Description  Database 

This  database  shall  contain  the  information  pertinent  to  describ- 
ing the  characteristics  of  the  documents  and  the  information  necessary 
to  retrieve  the  production/development  data  stored  in  the  other  two 

databases. 

Approximately  1.5  million  characters  (without  abstracts)  are  re- 
quired for  the  database.  If  the  abstracts  were  also  Included  in  the 
database,  there  would  be  an  additional  requirement  of  3 million  char- 
acters. 


6-3 


6.2.3  Historical  Database 

The  Historical  Database  shall  contain  susn&arircd  information  from 
the  productlon/dsvclopment  reports.  Approximately  1.4  million  charac- 
ters are  required  for  the  known  production/development  datasets  (Table 
6.2).  This  is  expected  to  grow  to  approximately  3 million  characters 
during  the  development  and  operation  of  the  pilot  facility. 

6.2.4  Complementary  Database 

The  Complementary  Database  shall  contain  detailed  information  from 
the  production/development  reports.  Approximately  6.4  million  charac- 
ters are  required  for  the  known  production/development  datasets  (Table 
6.2).  This  is  expected  to  grow  to  approximately  10  million  characters 
during  the  development  and  operation  of  the  pilot  facility. 

6.2.5  Output  Types 

The  major  pilot  facility  output  types  are  briefly  described  in  this 
subsection. 

1.  Historical  and  Complementary  Database  access 

Access  to  these  two  databases  shall  be  available  to  RADC  and  fac- 
ility personnel. 

2.  Data  Subscription 

Computer  readable  data  subsets  shall  be  available  for  distribution 
to  aid  in  research  efforts  that  require  productivity,  cost,  test, 
error,  change,  and  software  characteristics  data.  These  shall  be 
used  to  validate  and  refine  software  reliability  and  maintainabil- 
ity models  and  to  aid  in  additional  data  analysis  studies. 

3.  Data  Compendiums 

Reports  summarizing  the  database  holdings  shall  be  produced  as 
facility  publications. 

4.  A State-of-the-Art  Report  on  Testing  Tools  and  Techniques 

A report  on  various  methods  for  testing  computer  software  shall  be 
produced  Including  the  identification,  classification,  description, 
and  effectiveness  of  the  testing  tools  and  techniques. 

5.  A Software  Engineering  Glossary 

A comprehensive  glossary  of  software  engineering  terms  shall  be 
produced. 

6.  Document  Library 

Access  to  the  Document  Description  Database  and  the  source  docu- 
ments shall  be  available  to  RADC- and  facility  personnel. 


f Search  services  and  reference  information  dissemination  shall  be 

t extended  to  include  other  Government  Agencies  and  approved  contrac- 

£ tors. 
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The  facility  shall  not  act  as  a distribution  center  for  the  docu- 
aents  within  Its  collection.  Ordering  Information  shall  be  dls- 
acmlnatcd  to  thoae  requesting  documents. 

7.  Software  Engineering  Bibliography 

A bibliography  shall  be  produced  as  a facility  publication  and 
Include: 

- annoted  references  to  significant  literature 

• bibliographic  citations  for  each  reference 

• Indices  for  user  searches  Including 

concept  Index 

subject  term  Index  (selected) 
corporate  author  Index 

6.2.6  Analysis  of  the  Pilot  Input  Datasets 

The  five  known  source  datasets  for  the  pilot  facility  were  analyzed 
to  determined  data  content,  and  an  analysis  was  performed  to  determine 
the  data  requirements  for  representative  facility  outputs.  The  analy- 
sis technique  used  was  the  generation  of  an  input/output  matrix  listing 
group/attribute  names  for  the  SDR  databases,  an  Indication  of  the  con- 
tent of  each  source  dataset,  and  an  Indication  of  the  data  requirements 
for  the  output  products. 

In  Table  6.3  the  first  five  columns  reflect  the  data  content  for 
each  source  dataset,  the  last  four  columns  denote  the  data  requirements 
for  utilization  of  the  data.  The  sources  are  chose  listed  In  Table  6.1. 
The  following  is  a discussion  of  each  "use"  column  In  the  matrix  and 
should  be  considered  as  examples  of  facility  outputs. 

A)  Data  Compendiums 

Purpose  - Provide  summary  information  on  the  database  holdings, 

Input  Requirements  - Project  and  system  type;  person  and  computer 
type;  standards,  collection  procedures  and  technology  utilized; 
calendar,  person,  and  computer  time  and  cost;  change  attributes, 
quality  and  complexity  measurements,  and  constituent  attributes . 

B)  Reliability  Data  Subsets 

Purpose  • Provide  data  subsets  for  software  reliability  modeling 
efforts. 

Input  Requirements  - System  and  module  name  and  type;  constituent 
attributes;  testing,  error,  correction,  and  change  information. 

C)  Complexity  Data  Subsets 

Purpose  - Provide  data  subsets  for  calculation  of  complexity  meas- 
urements. 

Input  Requirements  - System  and  module  name  and  type;  constituent 
attributes. 


D)  Cost  Estimation  Data  Subsets 

Purpose  - Provide  data  subsets  for  cost  estimation  studies. 

Input  Requirements  • Project  and  system  type;  person  attributes; 
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constraints,  prlorltlas,  and  technology  attributes;  total  cost; 
amount  of  documentation  and  number  of  instructions  produced; 
quality  and  complexity  measurements;  constituent  attributes. 


This  list  of  data  requirements  is  considered  to  be  open  ended  be- 
cause of  the  need  to  provide  a variety  of  data  compendiums  and  data 
subsets. 

6.3  Pilot  Information  System  Development  and  Rccomaendations 

6.3.1  Introduction 

This  section  presents  the  recommendations  for  the  pilot  Information 
system  and  the  development  needed  to  establish  an  information  system. 

6.3.2  Developments  for  Input  Processing 

The  input  processing  procedures  discussed  in  Section  3 should  be 
followed.  However,  the  following  developments  must  first  occur. 

6. 3. 2.1  Construction  of  a Glossary  of  Software  Engineering  Terms 

So  that  there  is  a consistency  of  terminology  to  describe  items 
in  Che  database,  there  is  a need  for  Che  construction  of  a glossary 
of  software  engineering  terms.  As  part  of  the  present  study  effort, 
a glossary  of  software  engineering  terms  used  in  the  literature  was 
generated  (Appendix  C) . This  Glossary  should  be  refined,  and  in 
conjunction  with  the  work  now  being  accomplished  by  Che  IEEE  Sub- 
committee on  Software  Engineering  Standards,  be  used  as  a guideline 
for  indexing  documents  and  standardizing  Che  classification  within 
the  database  definitions.  In  addition,  this  glossary  will  be  pu- 
blished and  disseminated  as  a facility  publication. 

6. 3.2.2  Thesaurus  Construction 

Related  to,  but  distinct  from  the  construction  of  the  glossary, 
is  Che  need  for  the  construction  of  a thesaurus  for  document  index- 
ing and  retrieval. 

The  thesaurus  should  be  developed  using  the  following  proce- 
dures: 

1.  Construct  a sec  of  thesaurus  forms  as  illustrated  in  Figure  6.1 
using  the  keywords  (282)  and  the  papers  (59)  contained  in  The 
Proceedinss  of  the  International  Conference  on  Reliable  Soft- 

2.  Generate  a mini- thesaurus  consisting  of: 

- A structured  term  list  grouping  the  terms  into  logical 

categories 

- An  alphabetical  term  list  which  includes  ell  the  concept 
terms  of  the  structured  term  list 
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3.  Construct  an  additional  sat  of  thasaurus  forms  using  this 
mlnl-thasaurus  and  tha  documants  In  tha  prasanc  library. 

4.  Updata  tha  mlnl-thasaurus  to  ganarata  a thasaurus  that  will 
serva  as  a basis  for  subsaquant  documant  Indaxlng  and  ratrlaval. 

6. 3.2.3  Input  Forms  and  Data  Formats 

Standard  Input  forms  and  procaduras  must  ba  daslgnad  to  ba 
compatible  with  tha  databases.  Tha  attrlbutas  of  tha  databasa 
must  ba  dafinac  in  tha  data  definition  language  of  tha  databasa 

managamant  system. 

6. 3.2.4  Analysis  of  Input  Datasets 

Procaduras  must  ba  established  to  evaluate  tha  scope  and  con- 
tent of  tha  Input  datasets.  Tha  purposes  of  this  evaluation  are 
to  determine  the  applicability  of  tha  data,  to  determine  tha  re- 
quiramants  for  further  processing  to  transform  tha  data  to  tha 
SDR  formats,  and  to  determine  the  sensitivity  of  tha  data. 

6.3.2.5  Data  Translation 

All  of  tha  known  source  datasets  for  tha  pilot  facility  have 
soma  or  all  of  tha  data  in  computer  readable  form.  It  is  axpactad 
that  additional  datasets  will  also  ba  in  computar  readable  form. 

Procedures  must  ba  established  for  transforming  these  datasets 
into  the  SDR  databasa  formats.  It  is  racommendad  that  general  pur- 
pose translation  software  ba  developed  along  tha  lines  of  the  data 
translation  tools  developed  by  the  University  of  Michigan  Data 
Translation  Project. These  tools  provide  for  describing  both 
the  source  and  target  data,  and  the  necessary  transformations  re- 
quired to  generate  tha  target  data  from  the  source. 

For  those  datasets  that  arrive  In  hard  copy  form  and  contain 
ssMll  amounts  of  data  (such  as  the  system  productivity  and  cost 
data  from  the  MPP  Projects),  It  Is  recommended  that  manual  trans- 
lation be  performed  to  be  compatable  with  tha  SDR  databases. 

In  addition  to  conversion  of  these  foreign  files  Into  the  data 
format  for  the  pilot,  code  conversion  and  data  summarization  pro- 
cedures are  required. 

6. 3.2.6  Security  Procedures 

The  facilities  of  the  operating  system  and  the  database  manage- 
ment system  shall  be  used  to  provide  backup  and  restart  and  recov- 
ery capabilities  to  protect  the  database  and  the  application  pro- 
grams from  destruction  or  loss.  It  la  reconmanded  that  no  corporate 
sensitive  data  be  accepted  for  the  pilot  facility. 
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pf  ft. 3. 3 lyttM  ■•cowwiftattotia 

Froa  til*  •F*tra  •lc*rMtlv«*  yr***Bt*d  in  ft*ction  S.  cb*  following 
v*cow*ndatlon*  nr*  and*. 

ft. 3. 3.1  Dat*  Colltction  ftyataaa 

Aa  char*  la  no  eo*  data  collaction  ayatan  that  cellact*  a cm*- 
plat*  tat  of  data,  it  ia  r*co«and*d  that  data  fra*  varlOMt  data 

cellactlon  ayataaa  h*  ac^uirad  and  aaalwatad.  It  la  alae  r*eon> 
■andad  that  durlni  th*  daaaloynant  *f  th*  yilet  facility,  th« 
ayacif icationa  far  a data  collaction  ayata*  h*  lanaratad. 

ft. 3.3.3  Data  ftatry 

Aa  th*  najority  of  th*  known  aovrc*  dataaata  far  th*  yilat 
will  h*  in  conyutar  raadaftl*  farn,  tha  ratMlranant  far  havatrokina 

data  ia  niainal. 

Tharafora,  it  ia  racannandad  that  aniatint  kayyx^nchaa  ft*  waad 
to  eonvart  thl*  data  to  conpatar  raadaftl*  farn. 


ft. 3. 3. 3 Infernation  lyatar 

It  it  racomaandad  that  th*  RADC  Ncnawall  ftlftC  Canputar  ftvata* 
utini  th*  CCOS  oparatini  ayatar  and  th*  WK.S  dataftaa*  nanaft^ant 
ayatan  ft*  uaad  for  tha  inrlanantatian  and  ratriaval  of  th*  fatil* 
ity  dataftaaa*.  Thl*  altarnativ*  praalda*  aaay  t*  a**  pradattic- 
eriantad  hardwar*  and  aeftwar*  aarvicat  to  hath  ftADC  and  fatilit 
paraonnal  to  aaaiat  in  thair  raaaarch  affort*. 

It  ia  raconnandad  that  dvring  tha  oparatian  *f  th*  pilot  fatil* 
ity  th*  AJIPA  Katvcrk  not  ft*  waod  to  pravtd*  dataftaa*  rat  naval  *--.< 
updating  capaftilitia* . Howavar,  it  i*  racannandac  that  a to**  a* 
undartakan  to  datarnin*  th*  faaaiftility  of  waing  th*  AAIa  .•atwer* 
a*  th*  facility  axpanda  to  a fully  oporational  cantat. 

ft. 3. 3. ft  Docunani  Liftrary 

ftacaua*  of  th*  axpactad  nuaihar  of  docuawnta  in  th*  docunar.i 
library  and  th*  naad  for  a varaatil*  infomation  atorag*  and  ro- 
triaval  capability,  it  ia  raconnandad  that  th*  daacriptlon  of  th* 
docuaanta  b*  atorad  in  a eonputar-raadabl*  Docunant  Oaacription 

Databaa* . 

6. 3. ft  Data /Docunant  Arquiaition 

Son*  dataaata  and  taxtual  docunanta  hav*  boon  idantlfiad  to  fonr. 
th*  baaia  for  th*  pilot  facility.  Howavar,  to  inaur*  eontinuad  acqui- 
aitien  of  data  for  th*  pilot  and  bayond,  thar*  ia  a naad  for  th*  aata- 
bliahipant  of  a data  acquialtion  progratc.  Th*  nathoda  and  procadures 
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for  tho  idonclflcocioo  «nd  acqulslcioa  of  eh«  docuMtic  input*  Bust  b* 
••c*bU*ti*d  during  th*  davolopaanc  of  ch«  pilot  facility  to  aaaur* 
quality  data  acqulaitioa  froa  all  aagaanta  of  th*  govarnaant-induatry 
aoftwar*  davaiopaant  coaaunity. 

Procadura*  for  th*  acquiaition  of  tha  taatual  inforaation  can  ba 
accoapiiahad  routinaly  with  ainUMl  anginaaring  affort.  Th*  acquiai- 
tton  of  production/davalopaant  data  raqulraa  aora  datallad  procaduraa 
that  aaalat  In  idantifying  potantial  aourcaa,  aolicitation  of  thaa* 
aourcaa.  and  tha  acquialtion  of  th*  data  froa  thaa*  aourcaa.  iacauaa 
of  aconoaic  and  propriatary  conaidarationa,  thla  acquialtion  i*  aora 
difficult. 

b.«  Pilot  Facility  Oparatlon 

It  ia  racoaaandad  that  tha  raaponaibility  for  th*  oparation  of  tha 
Pilot  facility  b*  dividad  into  four  aajor  functional  araaa  (Figura  6.2). 

1.  TaciMiicai  and  adainiatrativ*  diraetion 

2.  Oac viaant  acquialtion 

).  Inforaation  pracaaaing 
4.  Data  and  infarnation  analyaia 

4.«.1  Tachnical  and  Adainiatrativ*  Oiraction 

rh*  tachaical  and  adainiatrativ*  diraetion  involvaa  th«  ovarall 
raaponaibility  far  all  contractor  oparating  and  policy  daciaiona.  and 
pravidaa  official  intorfac*  to  th*  RAOC  Projact  tnginoor  on  tachnical 

nattara. 

Tha  thro*  rooMining  functional  araaa  ar*  tochnically  oriontad  with 
aach  roporting  to  a pro]*et  aanagar.  Thaa*  functional  araaa  ar*  ac- 
countabl*  ta  aach  ethar. 

6. A. 2 3ata/DocvMnt  Acquialtion 

Tha  function  of  data/docuaant  acquialtion  during  th*  oparation  of 
th*  pilot  facility  ancoapaaaa*  tha  idantif ication,  aolicicationi  and 
acquialtion  of  data  and  docuaant*  and  th*  rafinaaant  of  th*  aathoda 
and  procadura*  for  thla  acquialtion  procoaa.  Thar*  1*  a functional 
aaparation  batwoon  tha  acquialtion  of  taxtual  inforaation  and  prcduc- 
tion/davnlopaant  raport*. 

6.4.3  Inforaation  Procaaaing 

Thor*  ara  thro*  aajor.  but  Intarralatad.  functional  araaa  includad 
for  inforaation  procaaaing. 

1.  Th*  input  procaaaing  function  includaa  doctaanc  raviav  and  indaz- 
ing,  docuaant  library  and  databaa*  updating,  and  tranaforaation 
and  auanarizacion  Of  th*  production/davalopaanc  raporta. 
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2.  The  database  administrative  function  includes  controlling  the  con- 
tents and  integrity  of  the  databases  by  establishing  and  modifying 
database  maintenance  procedures,  evaluating  the  utilization  and 
growth  characteristics  of  the  database,  restructuring  the  databases, 
monitoring  and  evaluating  the  security  procedures,  and  by  esta- 
blishing user  Interfaces  to  the  database  and  the  requirements  for 
data  transformations. 

3.  The  Information  services  function  involves  the  evaluation  of  users 
needs  including  dissemination  of  information  on  services  provided 
by  the  facility;  the  production  of  data  compendiums,  subsets  of 
the  database,  and  a bibliography:  assisting  in  search  and  retriev- 
al requests;  providing  answers  to  "simple"  user  inqueries  and  gen- 
erating requirements  for  in-depth  analysis  requests. 

6.4.4  Data  and  Information  Analysis 

This  function  Involves  performing  data  and  engineering  analysis 
and  producing  technical  monographs  and  state-of-the-art  reports  con- 
taining the  results  of  the  research  efforts. 

At  a minimum,  for  the  pilot  facility,  this  includes  producing  a 
report  that  summarizes  and  evaluates  the  capabilities  of  tools  and 
techniques  that  have  been  developed  for  testing  software. 

Other  efforts  that  should  be  considered  include  software  reliabil- 
ity and  maintainability  model  development,  and  data  analysis  studies 
on  software  complexity,  quality,  productivity,  and  cost  estimation. 

6.5  Pilot  Facility  ^lanagement  Plan 

The  recoaaended  schedule  for  the  development  and  operation  of  the  pilot 
facility  for  a two-year  period  is  illustrated  in  Figure  6.3. 

The  facility  staff  should  consist  of  both  professional  and  non-profes- 
sional personnel.  It  is  recommended  that  the  professional  staff  consist  of 
Individuals  from  the  following  personnel  categories: 

- (tanager 

- Senior  Analyst 

- Software  Engineer 

- Information  Scientist 

and  that  the  non-professional  staff  consist  of  clerical  personnel  and  tech- 
nicians. 

The  capabilities  of  the  manager  are  required  for  overall  administration 
and  client  coonunication. 

The  principal  function  of  the  software  analyst(s)  is  database  admini- 
stration. This  includes  the  establishment  of  the  database  formats  and  the 
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Figure  6.3  Pilot  Facility  Schedule 


3.  Bibliography 
Data  and  inforwation  analysis 

a.  Testing  tools  and  techniques  report 

b.  Modeling  ettorts 

c.  Data  analysis 


dataset  analysis  procedures,  the  establishment  and  evaluation  of  the  data 
translation  and  security  methods,  and  the  maintenance  and  overall  responsi- 
bility for  the  databases. 

The  software  engineering  function  Includes  the  production  of  state-of- 
the-art  reports,  technical  monographs,  and  the  data  collection  system  speci- 
fications. It  also  Includes  the  establishment  of  the  data  acquisition  pro- 
cedures for  the  production/development  data,  and  the  acquisition  of  the  data. 
The  software  engineers  shall  provide  consulting  services  for  the  Information 
scientist  and  the  senior  analyst,  develop  computer  programs  to  supplement 
the  database  management  system,  and  shall  have  the  overall  responsibility 
for  the  production  of  the  data  compendlums  and  the  database  subsets. 

The  Information  science  functions  Include  the  construction  of  a glos- 
sary and  a thesaurus,  the  establishment  of  the  Input  forms  and  procedures, 
the  establishment  of  the  document  acquisition  procedures  for  the  textual  In- 
formation, the  Indexing  of  the  documents,  the  production  of  the  Bibliography, 
providing  search  and  reference  services,  the  acquisition  of  the  textual  do- 
cuments, and  the  overall  responsibility  for  the  document  library. 

The  clerical  personnel  are  responsible  for  dally  typing  and  report 
typing,  some  Input  processing  functions,  order  processing  and  shipping, 
telephone  services,  travel  arrangements,  etc. 

The  technicians  are  responsible  for  computer  data  entry,  data  summari- 
zation, and  ordering  documents. 

6.6  Pilot  Facility  Evaluation 

The  evaluation  of  the  pilot  facility  Includes  the  determination  of  the 
effectiveness  of  the  facility  in  terms  of  responsiveness  to  the  user's  needs 
and  the  operational  flexibility  and  efficiency  of  the  facility. 

The  evaluation  of  the  Internal  operation  of  the  facility  must  consider 
the  amount  of  time  to  process  the  source  input  and  the  amount  of  time  to  an- 
swer user  Inquerles.  The  amount  and  quality  of  the  data  and  Information 
dissemination  must  also  be  a consideration. 

The  users  of  the  pilot  facility  shall  be  RADC  personnel,  facility  per- 
sonnel, and  approved  RADC  contractor  and  other  Air  Force  personnel.  The  re- 
sponsiveness to  these  users'  needs  can  be  determined  by  both  quantitative 
and  qualitative  measures.  The  quantatlve  measures  Include  the  number  of  re- 
quests for  data  subsets  and  reports,  the  number  of  Inquerles  and  bibliograph- 
ic services,  the  percentage  of  these  requests  that  could  be  fulfilled,  and 
some  measure  of  the  usefulness  of  the  services  and  products  In  terms  of  re- 
sources saved.  This  last  figure  Is  subjective  In  that  the  users  themselves 
must  determine  the  relative  savings  from  using  the  products  and  services. 

The  qualitative  measures  Include  feedback  from  the  users  and  potential 
users  concerning  the  quality  of  the  products  and  services,  and  what  addition- 
al services  they  requite. 
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This  kind  of  fssdbsck  is  useful  in  iletermining  vhat  additional  ser- 
vices should  be  provided  and/or  ocher  areas  that  could  be  included  in  a fully 
operational  center.  This  feedback  can  be  acquired  through  participation  in 
govemaent  and  industry  software  engineering  sysposiums,  conferences,  neet- 
Ings,  and  committee  activities. 

It  Is  recommended  that  the  development,  operation,  and  evaluation  of 
the  pilot  facility  proceed  in  such  a manner  that  a fully  operational  center 
can  evolve  to  provide  continuous  and  expanded  services  to  the  users.  As  new 
services  are  provided  and  internal  processing  functions  are  expanded,  they 
•hould  be  accomplished  with  minlmuin  disruption  on  the  operation  of  the  re- 
pository, whether  it  be  Che  pilot  facility  or  a fully  operational  center. 

6.7  A Fully  Operational  Center 

As  the  pilot  facility  evolves  into  a fully  operational  center,  the  user 
community  shall  expand,  the  scope  of  the  Information  shall  change,  and  the 
engineering  analysis  activities  should  encompass  more  software  engineering 
areas  which  will  result  in  a better  quality  and  greater  quantity  of  output 
services. 

6.7.1  The  User  Community 

It  is  expected  that  as  the  repository  evolves,  the  user  community 
will  encompass  more  members  of  the  software  development  community  in- 
cluding not  only  government  agencies  and  contractors,  but  the  private 
sector  ss  well. 

This  expansion  of  the  user  community  necessitates  the  need  for  the 
incorooration  of  a cost  recovery  program.  This  program  can  consist  of 
charging  Che  user  an  annual  cost  which  would  entitle  the  user  to  pu- 
blications issued  during  the  period  plus  a prescribed  number  of  direct 
Inqueries.  In  addition,  separate  pricing  for  individual  publications 
and  consulting  services  can  be  instituted. 

To  provide  existing  and  potential  users  with  an  awareness  of  tne 
services  provided,  Che  following  vehicles  can  be  used: 

1.  Brochures  listing  repository  publications  and  services  along  witn 
complete  pricing  and  ordering  information. 

2.  Newsletters  discussing  repository  services  and  other  software  en- 
gineering activities. 

3.  Workshoos  concentrating  on  specialized  software  development  pro- 
blems, trends,  etc. 

4.  Presentations  at  software  engineering  conferences,  technical 
meetings,  etc. 

6.7.2  Information  Concent 

The  content  of  the  databases  and  the  number  of  documents  in  the 
library  will  increase  as  the  repository  expands. 
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The  primary  Informaclon  content  of  the  databases  tor  the  pilot  ' 
facility  consists  of  testing,  error,  and  change  data.  It  Is  expected 
that  as  software  reliability  and  maintainability  models  are  developed 
and  validated,  the  need  for  the  data  will  decrease,  but  will  still 
exist. 

With  an  aggressive  data  collection  and  acquisition  program,  more 
cost  and  productivity  data  should  become  available. 

The  Initial  estimate  for  the  size  of  the  pilot  facility  databases 
Is  approximately  16  million  characters.  By  being  selective  In  the  ac** 
ceptance  of  data,  purging  obsolete  data,  and  Including  productivity 
and  cost  data  In  the  Historical  Database  at  only  the  project  and  system 
levels,  the  size  of  the  databases  should  not  exceed  100  million  charac- 
ters. 

Because  of  this  change  In  Information  content,  one  or  more  of  the 
following  activities  should  occur: 

1.  Database  reorganization 

2.  Addition  of  new  data  Items 

3.  Purging  obsolete  complementary  database  entitles  and  constructing 
new  complementary  database  entitles. 

4.  Data  summarization  modification. 

5.  Modifying  Input  processing,  dataset  analysis,  and  data  translation 
procedures. 

6.  Establishing  a secure  informaclon  system. 

6.7.3  Output  Services 

The  output  services  should  be  expanded  to  Include  Che  results  of 
more  In-depth  studies  than  were  produced  during  Che  pilot  facility. 

This  can  be  accomplished  because  of  Che  expanded  scope  of  the  data, 
and  because  we  will  have  learned  more  about  Che  software  development 
process  and  Che  techniques  to  analyze  the  data. 

Database  and  document  library  access  shall  continue  to  oe  available 
to  all  qualified  users.  Data  subsets  and  data  compendlums  shall  be 
produced  as  user  requirements  are  Identified  while  a handbook  on  de- 
signing and  developing  computer  software  shall  be  developed  to  assist 
both  Che  practitioner  and  the  researcher. 

Technical  monographs  and  state-of-the-art  reports  shall  be  pro- 
duced as  user  requlremants  are  Identified.  It  Is  expected  chat  they 
will  cover  the  following  areas: 

- cost  .ind  resource  asclmatlon 

- software  development  monitoring 

- quality  software  production 

- software  quality  measuras 

- software  complexity  measures 

- software  requirements  and  specifications 
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Tht  lnqutz7  and  consulting  aarviccs  shall  bccone  an  increasingly 
Bors  iaportant  function  as  the  user  cooBunlty  expands,  sore  data  be- 
coBss  available,  and  as  greater  expertise  exists  in  the  repository. 

These  services  Bay  take  the  fora  of  answering  questions  on  the  use 
of  a data  compendiua  to  long  tera  consulting  actions.  To  provide  qual 
ity  consulting  services,  it  shall  be  necessary  for  the  software  engi- 
neers to  specialise  in  certain  areas.  These  areas  Bay  be  reliability, 
Balntalnabllity,  testing  tools  and  techniques,  developnent  techniques, 
requirestent  and  specification  Bethodologies,  and/or  project  Bonitoring 

The  consulting  tasks  nay  include  Beasurenent,  estination,  and  pre- 
diction of  the  reliability  of  a software  system;  assist  in  determining 
requirements  and  test  plans;  conduct  broad  and  indepth  surveys  among 
government  and  industrial  organisations  to  seek  out  specialised  infor- 
mation and  data;  and  perform  qualitative  and  quantitative  studies  on 
estimating  the  resources  for  a software  development  project. 
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7.  Reconmcndatlont 

The  following  recomfflsndacions  are  a summary  of  che  recommendations  contained 
in  this  report. 

- Establish  a software  data  repository  to  provide  users  with  access  to  qual- 
ity, relevant,  software  development  experience  information  to  assist  in 
their  research  and  development  efforts. 

- Develop  che  repository  so  that  continuous  service  be  provided  to  che  users. 

- Include  both  production/development  data  and  textual  information.  Esta- 
blish three  databases  Including: 

o Document  Description  Database  containing  descriptions  of  che  docu- 
ments and  che  data. 

0 Historical  Database  containing  summarized  productivity,  cost,  en- 
vironment, test,  error,  and  change  data, 
o Complementary  Database  containing  detailed  test,  error,  and  change 
data  with  Che  capability  to  add  additional  entities. 

- Provide  che  following  services  and  outputs: 

o Database  and  Document  Library  access 
o Database  Subsets  and  Data  Compendiums 
o Technical  Monographs  and  Scate-of-che-Arc  Reports 

o Handbooks  and  a Bibliography 

o Inquery  and  Consulting  Services 
o Data  and  Engineering  Analysis 

- Establish  a pilot  facility  as  a subset  of  a fully  operational  center. 

The  following  are  che  recommendations  for  che  establishment  and  operation 
of  a pilot  facility: 

o The  facility  be  located  at  RADC  in  Rome,  NY 

o Limit  the  users  to  members  of  che  Air  Force  and  approved  contrac- 

tors and  agencies. 

o Accept  only  non-sensitive  data  including  RADC  purchased  datasets, 
o Construct  a glossary  of  software  engineering  terms  and  a thesaurus, 
o Establish  procedures  for  acquiring,  evaluating,  summarizing,  and 
transforming  tha  production/development  data. 

0 Acquire  data  from  various  data  collection  systems.  Establish  the 
specifications  for  a data  collection  system  for  the  repository, 
o Implement  the  databases  on  the  RADC  Honeywell  6180  computer  system 
using  the  GCOS  operating  system  and  the  MDQS  database  management 
system. 

o Do  not  use  the  ARPA  Network  for  the  pilot  facility.  Determine 
feasibility  for  using  the  ARPA  Network  for  a fully  operational 
center. 
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0 Produce  date  conpendluas  and  data  aubscts  containing  productivity, 
error  and  coat  data;  a bibliography  of  relevant  aoftware  engineer- 
ing literature;  technical  ■onographs  containing  the  reaulta  of 
modeling  and  data  enalyaia  efforts, 
o Evaluate  the  facility  end  determine  additional  uaer  requireaents. 

Evolve  into  a fully  operational  center: 

o Expand  the  uaer  community, 
o Extend  acope  and  content  of  the  data, 
o Expand  output  services. 
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DATABASE  MANAGEMENT  TERMS 


ACCESS 

The  ability  to  approach,  conmunlcate  with  (input  to  or  receive  output  from), 
or  otherwise  make  use  of  any  resource  of  an  automated  data  processing  sys- 
tem. 

ACCESS  CONTROL 

The  process  of  limiting  access  to  the  resources  of  an  automated  data  pro- 
cessing system  to  only  the  authorized  persons  or  (in  computer  networks) 
other  data  processing  systems.  Access  control  may  be  implemented  by  use 
of  hardware  devices,  software  routines,  operating  procedures,  people,  and 
various  combinations  of  these. 

ACCESS  TYPE 

The  kinds  of  access  which  may  be  authorized;  e.g.,  read,  write,  append, 
modify,  delete,  create. 

CONTROLLED  ACCESS 

The  property  of  a software  security  system  that  allows  each  properly  iden- 
tified user  to  access  information  and  resources  within  the  system  to  which 
he  or  she  is  authorized,  but  no  more. 

DATABASE 

a)  A generalized,  common.  Integrated  collection  of  data  which  fulfills  the 
data  requirements  of  all  applications  which  access  it,  and  which  is  struc- 
tured to  model  the  natural  data  relationships  which  exist  in  an  enterprise. 

b)  A collection  of  interrelated  data  items  processable  by  one  or  more 
programs. 

DATABASE  ADMINISTRATOR 

A person  or  persons  given  the  responsibility  for  the  definition,  organiza- 
tion, protection  and  efficiency  of  the  database  for  an  enterprise. 

DATABASE  DIRECTORY 

A collection  of  descriptors  of  all  units  of  data  that  are  available  to  the 
database  management  system,  these  descriptors  being  derived  from  the  data 
description  language  statements. 

DATA  CLASSIFICATION 

The  level  of  security  classification  assigned  to  data. 

DATA  DESCRIPTION  LANGUAGE  (DDL) 

The  language  used  to  describe  the  database,  or  that  part  of  the.  database 
known  to  a program. 

DATA- ITEM  OR  ATTRIBtTIX 

The  smallest  unit  of  namad  data,  an  occurrence  of  which  Is  a repreaentatlon 
of  a value. 
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DATA  INDEPENDENCE 

•)  Th*  concept  of  •cparatlng  the  definition*  of  logical  end  physical  data, 
such  that  application  programs  need  not  be  dependent  on  where  or  hov  phy- 
sical units  of  data  arc  stored. 

b)  The  ability  to  alter  the  content,  forvat,  structure  and  physical  char- 
acteristica  of  a database  with  a ainiaal  impact  on  existing  programs. 

DATA  INTEGRITY 

The  concept  that  all  units  of  dats  sust  be  protected  against  accidental 
or  deliberate  invalidation. 

DATABASE  MANAGEMENT  SYSTEM 

An  integrated  set  of  computer  procedures,  designed  to  fscllitate  storage 
and  retrieval  of  large  amounts  of  structured  data  in  an  orderly  fashion, 
by  providing  frequently  used  functions  to  s user  at  a level  not  requiring 
detailed  knowledge  of  the  data  structure  or  major  portions  of  the  proces- 
sing system. 

DATA  MAIUPUIATION  LANGUAGE  (DML) 

The  language  used  to  cause  data  to  be  transferred  between  the  application 
program  and  the  database. 

DATA  STRUCTURE 

The  logical  relationships  which  exist  among  the  units  of  data  in  a data- 
base and  which  arc  under  control  of  a database  management  system. 

ENTITY 

A person,  place,  thing  or  event  of  interest  to  the  enterprise,  and  about 
which  data  may  be  recorded. 

ENTITY  NAME 

The  symbol  by  which  a person,  place,  thing,  class  or  any  object  of  thought 
is  known. 

FIELD 

A specific  defined  area  of  a record  used  for  a particular  category  of  data. 
A group  of  character  positions  used  to  hold  s value. 

FILE 

A collactior  of  all  occurrences  of  s given  type  of  logical  record. 

FILE  MANAGEMENT 

A generic  tern  for  the  function*  of  creation,  insertion,  delation  and 
modification  of  records;  raorganislng,  sorting,  merging  and  other  processes 
commonly  performed  on  files. 
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HIERARCHICAL  STRUCTURE 

A mechod  of  conceptualizing  and/or  storing  fields,  records,  or  files  so 
that  more  than  one*-for-one  relationship  Is  defined  as  existing  for  a file, 
and  Che  actual  distinctions  among  hierarchical  levels  are  expressed  in  Che 
data. 

HOST  LANGUAGE  SYSTEM 

A database  management  system  that  Is  built  upon  Che  facilities  of  a pro- 
gramming language  and  Is  Identified  to  the  application  programmer  as  new 
tools  of  the  language  for  logical  and  physical  file  manipulations.  The 
Cools  are  embedded  In  the  host  language  (e.g.,  FORTRAN,  JOVIAL,  COBOL)  and 
are  accessed  usually  through  CALL  statements,  but  sometimes  by  extensions 
In  Che  language. 

INDEX 

An  ordered  reference  list  of  the  contents  of  a field  or  fields  of  a file, 
together  with  a reference  notation  for  Identification  or  location  of  each 
field  and  most  likely  other  related  fields.  For  example,  an  index  might 
list  each  value  of  a field  and  Che  location  of  every  record  containing 
that  value. 

INVERTED  FILE 

A storage  organization  In  which  an  Index  Is  provided  for  the  values  of 
each  type  of  data-ltem.  If  an  Index  Is  provided  for  every  data  type,  the 
file  is  called  totally  Invereed;  if  an  Index  Is  provided  for  only  some  of 
the  data  item  types,  the  file  Is  partially  Inverted. 

LOGICAL  RECORD 

The  collection  of  fields  as  a unit  of  data  to  be  processed.  Independent 
of  their  physical  environment.  Portions  of  the  same  logical  record  may  be 
located  In  different  physical  records. 

NETWORK 

One  or  more  collections  of  directed  relationships  between  three  or  more 
units  of  data  such  that  some  units  of  data  arc  considered  owners  which 
others  arc  members,  where  each  member  may  have  more  than  one  owner. 

OCCURRENCE 

A specific  Instance  of  the  value  of  a unit  of  data. 

PASSWORD 

A secret  word  or  a string  of  characters  which  either  Identifies  or  authen- 
ticates a user  or  a defined  system  resource,  such  as  a program  or  data. 

PHYSICAL  RECORD 

The  collection  of  fields  as  a unit  of  data  which  Is  stored,  retrieved,  or 
moved  as  one  unit.  The  unit  of  data  Is  defined  In  terms  of  Its  physical 
storage  qualities. 
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PHYSICAL  SEQUENCE 

As  order  of  record*  or  field*  In  * flic  thet  i*  the  eeffie  order  in  which 
the  record*  or  field*  ere  located  in  edjacent  phyeical  storage. 

KEOORD 

a)  A collection  of  related  value*  treated  as  a logical  unit  during  any  op- 
eration of  the  data  eanageaent  syateis  (e.g.,  during  data  collection,  pro- 
ccaaing,  or  output).  (Logical  Record) 

b)  A unit  of  data  to  be  placed  on,  or  taken  from,  a atorage  device  in  a 
single  operation.  (Physical  Record) 


RECOVERY  PROCEDURE 

The  actions  to  be  taken  in  restoring  a system's  computational  capability 
or  the  data  files  after  any  system  failure  or  penetration. 

RELATIONAL  MODEL 

A logical  model  of  data  in  which  records  of  a given  type  have  a given  num- 
ber of  (nonrepeating)  items.  All  Instances  of  a record  type  are  considered 
to  be  collected  together  in  a mathematical  set.  The  database  is  considered 
to  be  a collection  of  time  varying  relations  of  varying  degrees. 

RE0RGA.N1ZATI0N 

The  process  of  rearranging  the  relative  physical  placement  of  data-units 
in  the  database. 

REPEATING  GROUP 

A set  of  associated  fields  in  a record  or  in  another  repeating  group  vhich 
may  have  more  than  one  value.  Individual  values  for  one  field  are  asso- 
ciated with  individual  values  in  the  other  fields  (which  may  include  the 
null  value).  A repeating  group  containing  only  one  field  is  generally 
called  a multivalued  field. 

REPORT  GENERATOR 

A computer  program,  usually  included  in  a database  management  system,  that 
can  direct  the  production  of  formatted  output  reports  if  it  is  provided 
with  format  specifications,  input  file  detail,  and  other  information. 

RESTRUCTURE 

The  process  of  adding  to  or  deleting  from  the  types  of  data-units  and  data- 
relationships  represented  in  the  database,  of  rearranging  data-units  which 
•re  components  of  larger  data-units,  and  of  making  the  corresponding 
changes  to  its  schema. 

RETRIEVAL 

The  operation  in  a database  management  system  used  to  recover  data  stored 
in  a database  in  such  a way  that  it  is  available  to  other  database  manage- 
ment system  functions.  A retrieval  generally  involves  conditional  selection 
of  data  and  the  assembling  of  selected  records. 


RETRIEVAL  SPECIFICATIONS 


The  manner  and  syntax  of  specifications  used  to  express  the  conditions  un- 
der which  flies,  records,  or  fields  In  a database  will  be  selected  for 
further  processing  during  a retrieval  operation. 

ROLL-BACK 

The  process  of  reversing  recent  activities  of  the  system,  to  restore  some 
of  or  all  of  the  database  to  Its  state  at  a previous  point  In  time. 

SCHEMA 

A complete  description  of  the  database  In  terms  of  the  characteristics  of 
the  data  and  the  Implicit  and  explicit  relationship  between  data-unlts. 

SECURE  OPERATING  SYSTEM 

An  operating  system  effectively  utilizing  those  hardware  and  software  con- 
trol functions  necessary  to  provide  the  protection  appropriate  for  the 
value  of  the  Information  and  resources  managed  by  the  system. 

SECURITY 

In  the  computer  community,  the  realization  of  protection  for  hardware, 
software,  and  data. 

a)  Administrative  Security  — The  management  constraints,  operational  pro- 
cedures, accountability  procedures,  and  supplemental  controls  used  to  pro- 
vide an  acceptable  level  of  protection  for  secure  material 

b)  Computer  Security  — The  hardware/software  functions,  characteristics 
and  features  needed  to  provide  an  acceptable  level  of  protection  In  a 
computer  system. 

c)  Data  Security  — The  protection  of  data  from  accidental  or  Intentional 
but  unauthorized  modification,  destruction,  or  disclosure. 

d)  Information  Security  — The  hardware/software  functions,  characteristics, 
and  features;  operation  procedures,  accountability  procedures,  and  access 
controls  at  the  center  computer  facility,  remote  computer  and  terminal 
facilities;  the  management  constraints,  physical  structures,  and  devices; 
and  personnel  and  communications  controls  needed  to  provide  an  acceptable 
level  of  protection  In  a computer  system. 

e)  Personnel  Security  ~ Insuring  that  all  personnel  who  have  access  to 
any  sensitive  data  have  the  appropriate  clearances. 

f)  Physical  Security  — Physical  security,  as  It  pertains  to  computers, 
does  not  differ  from  physical  security  for  other  Installations.  It  Is 
achieved  through  the  use  of  locks,  guards,  badges,  personnel  security 
clearances  and  administratively  controlled  measure  outside  the  computer 
as  well  as  measures  required  for  the  protection  of  the  structures  housing 
the  computer  and  relat^  equipment  against  damage  from  accident,  fire 
and  environmental  hazard,  thus  ensuring  the  protection  of  their  contents. 
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SELECTED  RECORDS 


The  set  of  cooplcte  or  partial  records  which  are  the  result  of  a retrieval 
operation  and  becone  available  for  further  processing,  such  as  for  sorting 
and  printing. 

SELECTION  CRITERIA 

The  conditions  expressed  or  loplled  by  a user  which  are  used  selectively 
to  recover  data  during  a retrieval  operation:  i.e.,  a logical  expression. 

SELF-CONTAINED  SYSTEM 

A database  management  system,  the  capabilities  and  language  of  which  are 
Intended  primarily  for  the  nonprogrammer.  They  are  self-contained  in  the 
sense  that  they  usually  have  no  connection  with  any  procedural  language 
(except  that  the  system  Itself  may  be  written  In  a procedural  language, 
or  It  may  permit  user-written  code  In  a procedural  language). 

SENSITIVE  INFORMATION 

Information  whose  disclosure  or  modification  is  disadvantageous. 

SEQUENTIAL  FILE  STRUCTURE 

A method  of  storing  and  retrieving  data  in  which  information  becomes  avail- 
able in  a one-af ter-the-other  sequence  only. 


SET 

A named  collection  of  related  records  representing  a one-to-many  relation- 
ship between  the  owner  and  member  records. 

SORT  KEY 

A field  of  a record  whose  values  will  be  used  in  arranging  the  records,  or 
just  the  values,  into  a sorted  sequence. 

STORAGE  ^ND  RETRIEVAL  SYSTEM 

A system  for  storing  and  locating,  on  demand,  certain  documents  or  other 
records  relevant  to  a given  information  requirement  from  a file  of  such 
material.  Examples  are  classification,  indexing,  and  machine  searching 
systems. 

SUBSCHEMA 

A description  of  those  data-units  and  relationships  from,  a database  of 
intcrast  to  a particular  program. 

SUBSET  FILE  (also  Data  Subset) 

A selected  portion  of  a data  file  arranged  so  that  it  is  also  (or  can  be 
used  as  if  it  were)  a data  file,  separately  processable  by  the  database 
management  system. 
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Appandlz  B 
SURVEY  RESULTS 


ABSTRACT 

The  results  of  e survey  on  the  exchange  of  software  development  experience 
Information  are  presented  in  this  Appendix.  These  results  Indicate  that  mem- 
bers of  the  software  community  are  Interested  In  the  establishment  of  a center 
for  information  sharing  and  are  willing  to  share  their  experiences  and  data  on 
developing  computer  software.  They  arc  Interested  in  the  results  of  analyses 
on  determining  the  effects  that  different  developmental  and  testing  philoso- 
phies have  on  the  cost  and  productivity  of  software  development. 
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Introduction 


To  dtvtlop  the  tpoclflcctlona  for  th«  Software  Data  Rapoaltory,  it  vaa 
necaaaary  to  detarmina  tha  potantial  uaara  of  tha  data  cantar  and  tha  aoftvara 
anginaaring  araaa  in  which  thay  ara  intaraatad.  To  aacartaln  tha  typaa  of 
potantial  uaara  and  thair  tachnological  intaraat,  wa  parforaad  a aurvay  of  a 
portion  of  tha  aoftmra  davalopmant  coomtunity. 


Bacauaa  a aoftwara  axparianca  databaaa  would  ba  tha  haart  of  tha  data 
rapoaitory,  tha  aurvay  alao  containad  quaationa  concaming  tha  data  collaction 
afforta  of  tha  racipianta  and  thair  opiniona  on  aharing  thair  data  and  thair 
expariancaa  in  coilacting  tha  data. 


Thia  appendix  ia  organized  aa  followa.  Firat,  tha  background  for  tha 
survey  is  presented  followed  by  an  initial  analysis  of  tha  aurvay.  Thai^  tha 
results  of  further  analysis  la  praaantad,  correlating  response  categories. 

2.  Survey  Approach 

During  the  aunmer  of  1975,  a survey  was  sent  to  selected  members  of  tha 
software  development  community.  A total  of  1659  copies  of  the  survey  were 
mailed.  Two  mailing  lists  vara  used: 

1.  Recipients  of  tha  RADC-sponsored  "Structured  Prograsming  Series"  publi- 
cations RADC-TR-74-300  (875  names) . 

2.  Attendees  of  the  International  Conference  on  Reliable  Software  April  1975 
(784  names) 

By  tha  end  of  October,  337  responses  had  been  received  — a response  rate  of  22Z. 

The  response  card  used  for  this  survey  is  reproduced  here  as  Figure  1. 

The  questions  on  this  survey  card  contain  four  major  areas  dealing  with: 

A.  Usage  level  of  the  repository 

B.  Data  collection  activities  and  cooperation  posture 

C.  Tha  characteristica  of  the  respondees 

D.  Interest  level  for  tha  rapoaitory 


3.  Survey  Results 

Tha  survey  rasults  are  diseuased  below  relative  to  each  of  tha  quastions 
contained  on  the  survey  fora.* 

For  a more  detailed  analyaia  of  the  raaults,  see  Duvall,  L.M. , "Software 
Data  Repository  Study  - Survey  Raaulta",  IITRI  Report  E6330-5,  February  1976 
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Th«  survey  qusstieo  to  docorBino  tht  tisag*  inttrost  vss  In  Mtrlx  fon. 
Tht  revs  doslgnatsd  • technology  erce  (Rslleblllty  Prediction,  Testing  Tech- 
niques, Development  tools.  Other)  and  the  coluans  designated  e usage  level 
(Date  for  your  evn  Analysis,  Reports  on  Analysis  Tools,  Results  of  Analysis). 
Contained  In  Table  1 Is  a siaanary  of  these  responses.  Zn  the  first  three 
eeltans,  the  mabers  repreecnt  the  nuaber'.of  tines  that  cell  vas  checked; 
the  "Z  of  Total"  repreeents  the  percentage  of  survey  cards  that  contained  a 
cheek  nark  In  that  cell.  For  exanple,  136  survey  cards  contained  a check 
•ark  for  the  cell  "Reliability  Prediction,  Data  for  Tour  Own  Analysis".  Of 
all  cells,  this  cell  was  checked  the  least  nunber  of  tines  whereas  "Testing 
Techniques,  Results  of  Analysis"  vas  checked  nost  often  (226  tines). 

Figure  2 contains  hlstograns  representing  the  percentage  of  respondees 
that  checked  each  individual  cell.  In  all  three  technology  areas,  "Data  for 
Tour  Own  Analysis"  vas  checked  the  least  number  of  times  (40Z  for  Reliability 
Prediction,  47Z  for  Testing  Techniques,  and  482  for  Development  tools),  and 
"Results  of  Analysis"  vas  checked  the  greatest  number  of  times  (652  for  Reli- 
ability Prediction,  672  for  Testing  Techniques  and  662  for  Development  Tools). 

As  shown  in  Figure  3,  752  checked  one  or  more  cells  in  the  "Reliability 
Prediction"  row,  642  checked  one  or  more  cells  in  the  "Testing  Techniques"  row 
and  812  checked  one  or  more  cells  in  the  "Development  Tools"  row. 

Thirty-four  percent  of  those  that  responded  included  comments  in  the 
"Other"  row.  The  keywords  most  often  mentioned  or  implied  here  were:  cost, 
management,  reliability,  productivity  development  and  production  design  langu- 
ages, testing  standards,  and  maintainability. 


Data  Collection 

Sixty-evo  percent  of  the  respondees  indicated  they  were  collecting  data 
(Figure  4),  772  indicated  they  would  be  willing  to  share  experiences  (Figure  5), 
and  762  indicated  they  would  support  making  the  data  available  (Figure  6). 

Of  those  respondees  that  indicated  they  were  collecting  data,  832 
stated  they  would  be  willing  to  share  experiences  and  822  indicated  they  would 
support  making  the  data  available  (Table  II) . Of  thoae  that  were  not  collecting 
data,  662  noted  that  they  would  ahare  the  data  (i.e.  the  data  they  did  not 
have) . This  anomoly  most  likely  occurred  because  the  respondecs  wanted  to  in- 
dicate that  they  would  be  willing  to  ahare  if  they  did  have  the  data. 

Seventy-five  eosmients  were  mads  in  this  section  even  though  there  vas  no 
specific  "ether"  category.  Under  the  eollectiot  question,  the  majority  (702) 
of  the  comments  were  concerned  with  qualifying  their  collection  (e.g.  planning, 
isiformally,  some,  etc.).  In  addition,  six  specifically  noted  cost  data. 

For  the  ether  two  questions  on  sharing  mxperiences  and  data,  the  majority 
(752)  of  the  coBBents  had  to  do  with  the  need  for  approval  and  proprietary  lim- 
itations , 
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Of  tho«*  that  responded,  2SZ  steted  they  were  from  the  govemaent,  43Z 
from  industry,  20Z  from  universities  end  12Z  "other"  (Figure  7).  The  "other" 
cetegory  memos  thmt  either  none  or  more  then  one  box  wes  checked.  If  more 
then  one  box  wes  checked,  the  respondee  is  most  likely  teeching  end  consulting 
for  the  government  or  industry,  or  trorking  for  en  industriel  firm  on  govern- 
ment contrects. 

To  provide  e comperieon  of  those  who  responded  to  those  who  were  reci- 
pients of  the  survey,  the  meiling  lists  were  enelyzed  by  cetegorizing  the  in- 
stltutionel  type  of  the  recipients  (Teble  III).  Most  of  the  recipients  were 
not  difficult  to  cetegorlzed  except  for  the  "other"  cetegory.  As  cmn  be  seen, 
the  number  thmt  responded  followed  closely  to  those  who  received  the  survey. 

The  survey  elso  esked  the  potentiml  users  to  indicete  their  principel 
functions  as:  design  software,  software  development  reaearch,  and/or  evaluate 
software.  The  highest  percentage  (64Z)of  respondees  noted  that  one  of  their 
principal  functions  was  designing  software;  Che  lowest  percentage  (41Z}  indi- 
cated a user  of  software  (Figure  8). 

Receiving/Providing  Further  Information 

The  last  question  on  the  survey  was  included  to  provide  us  with  some  in- 
dication as  to  interest  in  the  repository.  The  response  was  overwhelmingly 
positive.  Of  chose  that  responded,  307  (or  91Z}  indicated  "yes"  to  the  ques- 
tion: "Are  you  interested  in  receiving  and/or  providing  further  information?", 

13  (cr  4Z)  indicated  "no",  and  17  (SZ)  checked  neither  box. 

Usage  vs  Institution  Type 

The  data  on  institution  type  as  related  to  potential  usage  shoved  chat 
Che  respondees  of  the  universities  were  slightly  more  interested  in  development 
tool  effects  chan  chose  from  the  government  or  industry.  Those  from  the 
government  expressed  more  interest  in  obtaining  the  reaulcs  of  analyaes  chan 
Chose  from  industry  or  the  universities.  Table  IV  contains  summary  data  cor- 
telating  this  potential  usage  of  the  data  repository  and  the  Institution  Type. 
For  example,  36  respondees  indicated  they  were  from  the  government  and  were  in- 
terested in  "Reliability  Prediction,  Data  for  Your  Own  Analysis".  Referring 
to  Table  I,  36  of  the  136  vho  indicated  interest  in  "Reliability  Prediction, 

Data  for  Your  Own  Analysis"  were  from  the  government.  In  reviewing  this  column, 
123  cells  out  of  a total  of  490  cells  (2SZ)  checked  by  the  government  re- 
spondees  indicated  interest  in  data  for  soma  technology  area.  This  is  the 
lowest  percentage  for  any  category  vs  institution  type.  However,  there  is  lit- 
tle variation  when  comparing  the  total  percentages  to  the  individual  percen- 
tages for  each  institution  type. 

Data  Collection  vs  Institution  Type 

Within  institutions,  the  greatest  percentage  C69Z}  collecting  data  are 
from  Che  government;  the  least  percentage  C48Z}  collecting  data  are  from  the 
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u&ly«rslti«s;  and  the  reepondcee  tram  the  unlvcreltlee  are  the  aoet  vllllng 
to  aharc  axpcrience  aad  date  (88X  and  65Z,  raepectlvely).  Although  atlll  an 
encouraging  raeponac,  thoae  froa  Induatry  ware  leaat  villlng  to  ahare  experl- 
aneaa  C70Z)t  The  dlatrlbution  of  the  reapondeea  by  Inatltutlon  type  to  thoae 
raaponding  "yea"  to  the  data  collection  queation  ia  illuatratad  in  Figure  9; 
to  thoae  raaponding  "yea"  to  the  aharing  experience  queation  la  ahown  In 
Figure  10;  and  to  thoae  reaponding  "yea"  to  the  aharing  data  queation  ia  illus- 
trated in  Figure  11. 

Function  va,  Inatitution  Type 

The  highaat  percentage  of  reapondeea  that  atated  they  vere  designing 
aoftvare  was  iron  industry  C72Z)  and  the  highest  percentage  of  reapondeea  for 
isplcaenting  software  (692)  was  also  froa  industry.  The  govemaenc  reapondeea 
shoved  a slight  edge  over  Industry  (49Z  vs.  462)  for  evaluating  software;  the 
universities  shoved  a alight  edge  over  industry  (442  va  422}  as  users  of 
software  (Figure  12). 

The  largest  differential  occurred  in  the  aoftvare  developaent  research 
area.  The  universities  Indicated  712  compared  to  612  for  "other",  502  for 
industry  and  492  for  govemaent. 

Interest  vs  Institution  Type 

As  seen  in  Figure  13,  912  of  all  the  respondees  indicated  they  vere 
interested  in  receiving/providing  further  information.  The  highest  per- 
centage of  affirmative  responses  came  from  industry  (942).  The  lowest  per- 
centage of  negative  responses  came  from  government  (22). 


4.  Conclusions 


First  and  foremost,  the  survey  results  Indicate  that  the  software  develop- 
ment community  ia  interested  in  the  establishment  of  a software  data  reposi- 
tory. The  potential  users  are  most  interested  in  receiving  information  on  the 
affects  that  different  tools  and  techniques  have  on  the  software  development 
process;  and,  secondarily,  arc  interested  in  hsvlng  access  to  data  so  that  they 
can  perform  their  own  analyses.  The  effects  on  productivity,  costs  and  reli- 
ability are  alao  areas  of  user  interest. 

Most  of  the  potential  users  are  either  collecting  data  or  planning  to  do 
so,  and  an  overwhelming  majority  are  villlng  to  share  these  data  with  a non- 
partisan organisation.  The  user  group  shoving  the  highest  level  of  data  col- 
lecting activities  is  the  government,  rather  than  industry  or  the  universities. 
Vnivarsity  pereonnel  are  the  most  willing  of  the  three  user  groups  to  share 
thair  data. 

The  principal  functions  of  govemaent  and  industrial  users  are  designing 
and  Implementing  software;  whereas,  with  the  universities,  the  principal  func- 
tion is  performing  research  in  the  area  of  software  development. 
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1.  Introduction 

This  Appendix  contains  definitions  of  software  engineering  terms  as  they 
appear  in  the  literature,  A list  of  the  papers  and  reports  that  were  searched 
and  used  for  the  term  definitions  is  included  in  Section  2,  along  with  the 
corresponding  reference  number. 

Section  3 contains  a list  of  the  terms,  the  reference  number  (in  paren- 
thesis) of  the  document  that  contained  the  definition,  and  the  definition. 


2.  l«f*rcnc*s 

001  TRW  Systens  Croup,  "Software  Reliability  Study",  RADC-TR-74-250,  October 
1974,  AD  787  784 

002  Shooman,  Martin  L. , "Software  Reliability:  Meaeureaent  and  Models",  Pro- 
ceedings 1975  Annual  Reliability  and  Maintainability  Syaposiim.  Jan.  1975, 
pg.  485-491 

003  Reifer,  D.J.,  "Interin  Report  on  the  Aids  Inventory  Project",  Report 
SAMSO-TR-75-184,  16  July  1975 

004  Triable,  John  T.,  "Structured  Progranming  Series  Volume  IV,  Date  Struc- 
turing Study",  RADC-TR-74-300,  21  April  1975,  (A015794). 

005  Barry,  Barbara  S,,  "Structured  Prograoalng  Series,  Voltase  X,  Chief  Pro- 
graaaer  Teas  Operations  Description",  RADC-TR-74-300,  22  January  1975, 

AD  A008  861 

006  Smith,  Ronald  L,,  "Structured  Prograosing  Series,  Volume  IX,  Management 
Data  Collection  and  Reporting",  RADC-TR-74-300,  24  October  1974, 

AD  A008  640 

007  Kessler,  Marvin  M. , Kister,  Vllliam  £,,  "Structured  Programming  Series 
Volume  XIV,  Software  Tool  Impact",  RADC-TR-74-300,  22  May  1975,  (A015795) . 

008  Kraly,  Thomas  M.,  et.  al.,  "Structured  Programming  Series,  Volume  VIII, 
Program  Design  Study",  RADC-TR-74-300,  30  May  1975  , (A016415). 

009  Smith,  Ronald  L.,  "Structured  Programming  Series,  Volume  X\’,  Validation 
and  Verification  Study",  RADC-TR-74-300,  May  1975,  (A016666). 

010  Manley,  John  H. , "Embedded  Computer  System  Software  Reliability",  Defense 
Management  Journal,  Vol,  11,  No.  4,  October  1975,  pgs.  13-17 

011  Wagoner,  W.L.,  "The  Pinal  Report  on  a Software  Reliability  Measurement 
Study",  Report  No.  TOR-0074(4112)-1,  15  August  1973,  Contract  No. 
F04701-73-C-0074 

012  Bratman  et.  al.,  "The  SDC  Software  Factory",  1 August  1973,  TM-5075/100/00 

013  Ross,  Douglas,  T. , Coodenough,  John  B, , Irvine,  C.A.,  "Software  Engineer- 
ing: Process,  Principles  and  Coals",  Computer.  Hay  1975 

014  Schwartz,  Julea  I.,  "Construction  of  Software:  Problems  and  Practicali- 
ties", Practical  Strategies  for  Devt loping  Large  Software  Systems.  Ellis 
Borovitz,  Ed.,  Addison  Wesley  19^5 

015  Boehm,  Barry  W. , "Software  Design  and  Structuring",  Practical  Strategies 
for  D^eloping  Large  Software  Systems . Ellis- Horowitz  Ed.,  Addison-Wesley 
1975 


C-2 


016  Bocha,  B.U. , ct.  al.»  "Charactcrltcics  of  Software  Quality",  TRV-SS-73-09, 
28  Dae  1973 

017  Clapp,  J.A. , "Software  Engineering:  Problems  and  Future  Developments", 

Nov.  1974,  MTR-2791,  ESD-TR-74-195 

018  Lupplno,  F.M.  ct.  el.,  "Structured  Programming  Series,  Volume  V,  Pro- 
granalng  Support  Library  (PSL)  Functional  Requlreaients",  RADC-TR-74-300, 

25  July  1974,  AD/A  003  339 
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Aecr«att«tten  (0I2> 

All  tetlvitits  ts«t«  ttk«n  tOQftStr,  vstatlish  • tuffleltnt 
itvil  of  eonftotriet  in  ftn«l  preou't  tM  otvtleoar 

ts  ablt  to  gutrant**  Its  funettenal  otrforxtnet  to  sptelf iestlens 
ana  to  proviat  a tarranty  to  tna  cjiionsr  **ltn  alnimua  risk  ot 
asjitlonal  support  at  tnt  oavalapor's  asptnst. 

Accuracy  stuay  procassor  IOOJ> 

A cpy^Duttr  pragma  usto  to  atrrorn  ealeulotlont  to  assist  In 
astsrminlng  if  pregrsn  varlablts  art  eoaputta  wim  rtpulrtd 
accuracy. 

Actual  Oita  (OtQ) 

Oita  oascrtblng  tht  rasults  of  pregrsctnlng  actions  for  a 
projaci  that  will  ba  tna  primary  oata  ineluaad  In  tha 
Ttanagamant  reports. 

Anal/cicsl  ''PUallng  (003) 

i'na  t^a  tachnlrut  ustu  to  ayprtss  msthamati ea lly  (usually  by 
s sat  of  aauatlons  a raprasantatlon  of  son*  rial  problem, 
uuen  nojils  are  valuahla  for  atstraetlng  th«  assenca  of  tna  subject 
of  Inquiry,  because  equations  oescrlbin;  complex  systems 
tanj  to  become  compllcsteo  ano  often  Impossible  to 

formulite.  It  is  usually  necessary  to  make  simplifying  assumptions  wnich 

nay  olstort  accuracy,  soeclflc  lannuaga  ano  simulation 
systans  serve  as  aids  to  Inplenentatlan. 

.■'nal/tar  (003) 

A conp’jter  orenram  used  to  orovlse  source  langjage  or 
e:'»cjtlon  freruaney  statistics  at  tme  orogrem  or 
so  ?rce-siaia-.ient  level  to  assist  In  perfornence  evaluation  and  ' 
jcternimtlon  of  test  esse  coverage. 

>.sten:.lers  (012) 

I'ools  that  translate  oroprens  «»rltten  in  symbolic  machine 
len-^uece  Into  actueJ  mecnine  Isnjunpt  programs. 

.‘uto-eteo  net-'orV  or  path  snelvsls  (009) 

\ tcehnln-je  whle"  oeflnes  a orictlcsl  meesureble  means  by  axsminlng 
source  cooe  ano  oeterrlnlnn  the  minlnun  set  of  peths  ehlch 
exercise  all  loglcil  trenches  of  a progrsn. 

Autometej  test  generj-tor  (003) 

A compiler  propren  tnst  eerepts  inputs  soeclfytng  a test 
scenario  In  sons  soeelel  languege,  nenerntes  the  exact 
co'.ojter  Inputs,  enj  peierrlnes  me  expected  results. 

Autenntaa  verification  systems  (003) 

.inrpjier  progrens  that  Instrunent  the  source  eooe  by 
nenerstlng  anc  Inserting  counters  et  strategic  points  to 
prpvipe  e.aesures  of  test  effectiveness,  ihey  proviue  dets  that 
uAtBlls  hpu  thoroughly  the  source  eooe  has  been  exercised. 
jloc<  ilegrsn.  (OOi) 

1 ulenram  of  s systen,  Instrument,  or  computer.  In  «»hlch 
me  princlole  osrts  are  represented  oy  suitability  assoeieted 
lepnetrlcnl  figures  to  show  both  the  basic  functions  snd  the  functional 
rilstipnships  among  the  oarts.  (A.'Sl) 
euogtting  anu  estlnating  (012) 

ihose  actlvitlts  that  determine  the  levels  of  effort  and 
resources  neeoed  to  accomplish  a project. 

Bug  (002) 

one  or  nora  softwsre  buge  exist  in  s system  if  a software 
Change  is  regulrao  to  correct  a single  nsjor  error  or  nlnor 
error  so  ss  to  mast  spacifUd  or  Impllao  tystam  perfonesnet  raoulremants. 
Certification  (009) 

Csrrits  tha  eonnotstlen  of  on  authorl tatt va  endorsement  and 
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a«*ns  to  l-’oly  t*stlfyln5  In  vrltlnq  t^at  th*  orogran  ts  of  a 

csrtaln  standard  or  'Quality.  Certification  usually  Inplles  tha  existence 

an 

InJsoendent  quality  control  ^^rouo.  Certification  extends  the 
orocass  of  verification  ana  validation  to  an  ooeratlonil  envlronnent 
-iHw  Involves  acceptance  testlnq  of  the  overall  syste.x. 

Certification  (012) 

ihe  arPcess  of  oenonstratlnq  that  the  systfn  neets  Its  custoner 
rpTulranents  end  yjaranteelnj  that  cowollance  In  writing. 

Certl  fl  cation  tools  (012) 

l.n.-ornatlon  reporting  and  su’*narlzlng  conoonents  that 
provide  a systenetlc  uita  collection  and  sunnar Ir ing  nechanlsm 
th-'t  oroouces  a systen  status  report. 

C'nanna  (002) 

\ny  •Itorntlon  (aodltlon.  deletion,  correction)  of  the  proqraei 
code  whether  It  h*  a slnple  character  or  thousands  of  lines  of  code. 
C.h?nn»s  "aae  to  Inorova  docunentatlon  or  sitls/y  new  specifications 
ar*  Ihoortant  to  record  ana  study  but  are  not  counted  as  bugs. 

Chief  nro ^ra.'.T.ar  tee-  (017) 

r-ls  concept  inolles  the  use  of  e highly  structured  team 
of  sojcimists  for  software  oroouctlon  relying  on  technical 
oroceiuros  which  ere  cise-i  on  structured  prograimlng 
orlnclplos,  and  office  roceoures  bac'<aJ  up  with  autonated 
al-s  to  sunoort  grouo  * jnrunl cetlon . Specialized  roles  for 
oeoole  on  a project,  ana  tha  relationships  a.nong  than,  are 
1 1-jef Ined, 

Co'oar’tor  (003) 

f co'Tjter  progran  used  to  co-oare  two  versions  of  the  seme 
cfj'jtsr  orogran  under  test  to  establish  Identical 
cpi-.f  I Tintlon  or  to  specifically  Identify  changes  In 
th"  spjrce  codlnc  between  the  two  varslons. 

Cp-.oller  (CI2) 

^ tool,  useJ  In  the  oroajctlon  of  software  systens,  that  allows 
pro.Tihs  to  he  written  In  hi g.her-ordar  languages. 

— )lss  Ihcludei  ?L/1  conoller,  rOSTSt.'/  Conplier, 

o. hj  codOL  coipiler. 

Co-p-jt-’r  01  th  (010) 

K ro;.rissntetlon  of  facts,  concepts  or  Instructions 

In  a structured  forx  sultaoie  for  accaota.hce,  Interoretatlon 

or  processing  hy  eonnunlcatlon  between  conputer  epulpnent. 

Such  (eta  can  he  external  (in  co-puter-reidafcle  form) 
or  resident  within  tha  computer  equipment  ana  can  h#  In  tha 
form  of  .inelog  or  algltal  signals. 

Cpnp*icnr  iqulpnant  conouter  hardware  (010) 

devices  eijible  of  aceeotlng  and  storing  computer  data, 
jxjcutlng  a systematic  sequence  of  ooeratlons  on  computer 
dhth  or  proQuclnn  conputer  outputs.  Such  devices  can  oerform 

p. ih  stent  111  interpretation,  computation,  conninlcatlon, 
control  end  other  logical  functions,  Sxamplesi  central 
processing  units,  tar.mlnels,  printers,  analog/dlgltal 
converters,  tape  orlves,  dls'cs  and  drums. 

cpmoutar  program  certification  f<X)3) 

ihe  tpst  ano  evaluation  of  the  complete  conouter  program 
ei-ed  It  ensuring  operstlonel  effectiveness  and  suitability 
with  respect-  to  mission  raqulrements  anaar  realistic  operating 
condl  tl  ons. 

Computer  program  validation  (003) 

Tna  test  enu  evaluation  of  tha  conplata  program  aimed  at  ensuring 
cp-pl lance  with  oerfornanca  on-d  design  criteria. 

Conputar  program  verification  (<X)3) 

ihe  Iterative  process  of  datarmlnlng  n^ethar  tha  product 
of  each  step  of  tha  computer  program  acgulsltlpn  process 
fulfills  all  raqulrenants  levied  by  tha  previous  step. 

Computer  software  (010) 

K conhinatlon  of  associated  computer  programs  and  data  . 
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r«qulr*(i  to  command  th«  eomputor  aoulomant  to  oorforra 
co-sputa tlonal  or  control  functions. 

Computtr  system  (010) 

tn  Intarsctlng  stscnbly  consisting  of  computer  o:)ulpmont 
computer  programs  sno  comouter  data. 

Computer  turnaround  tine  <0lfl» 

tna  tine  differential  between  the  subnlttel  of  a Job  to  the 
Co-souter  Center  and  the  return  of  the  Job  results  to  the 
progranner. 

Conflagration  control  (012) 

t methodology  concerned  with  procedures  for  controlling  the 
contents  of  a soft-«are  system,  a way  of  monitoring 
thi  stetus  of  system  components,  preserving  the  Integrity  of 
releasee  and  developing  versions  of  a software  system,  and 
controlling  the  effects  of  changes  throughout  the  system. 

Confi -juratl  on  management  (012) 

All  activities  releted  to  controlling  the  contents  of  a software 
system.  It  monitors  the  status  of  system  components, 
preserves  the  Integrity  of  released  and  oeveloolng  versions 
of  a system,  and  controls  the  effects  of  changes  throughout  the 
system.  It  Is  a process  dealing  as  much  with  oroceouras  as  with 
tools. 

Constants  auto  cheeVer  (003) 

A computer  program  used  to  sasreh  a tape  for  all  constants  and 
oeramtters  to  loentlfy  th#  nine  of  the  constant.  Its 
storage  location,  ano  the  binary  scale  factor.  These  are  then 
cOToered  with  specification  values  to  assure  eonpllance. 

Control  (012) 

A major  sub-oivlslon  without  conf iguretlon'menegenent.  The 
procedures  by  which  e.hpnnes  to  the  Design  eg ul r aments  are 
proposed  and  formally  orocetsto. 

Conversion  aids  (007) 

ih-3$e  soft-eare  tools  v.-nieh  tsilsi  In  converting  operational 
software  fron  one  compiler  to  another.  These  tools  analyse 

tn»  source  coot  as  written  for  one  compiler  and  hlgnllght  those 
st-^tenents  wnieh  are  not  compatlole  wiin  the  ceosbllltles  of  the 
ternet  compiler.  In  some  Instencee  t'*es»  conv'rslon  alas  will 
re.iece  the  Incompetl-le  steiememts  witm  one  or  more  tsrget  conpiler 
stetemeni  which  ere  designed  to  acnleve  the  saiie  rttult. 

Correctness  proof  (OOu) 

me  tachr.inue  of  proving  mcthenetlcsl ly  thst  • given  program  Is 
coislstent  wit''  e given  set  of  specif Icstions.  This  process  can  be 
eccompllshed  by  manual  netnods  or  ny  program  verifiers  requiring  manual 
intervention. 

Correctness  proeft  (003) 

Auto-'aie.)  verification  systems  exist  which  allow  th# 
e''glyit  to  prove  snsll  pronrans  are  correct  by  means 
sl"ller  to  thost  ustu  in  proving  methemstlcal  theorans. 

*-ulon  anj  thaorems  otrlveo  are  used  to  estscllsh  validity 
of  orogrnn  assertions  ems  to  provide  a funaamental  understanoing 
of  ho'w  tha  program  ooeratas. 

Cost  ueta  (t>l4> 

Dnta  describing  the  costs  assoclattd  with  the  rtsources 
expendeo  by  a soft'vart  project. 

Cost  estlmstion'  (012) 

A ttanoard  tachnlqut  for  astlnstlng  the  amount  of  labor 
ntetsssr-y  for  th#  conplatlon  of  a tasy,  the  amount  and 
ootanttal  costs  of  eonputtr  time  required,  etc.,  prior  to  eno 
ouring  e project's  llfatlne. 

..  Cost  rssnegenent  (012) 

A collecteo  set  of  tools  that  provide  the  crlterle  end  devices 
^ for  trecklng  project  costs, 

t Cross  compiler  (012) 

A tool  that  can  operate  on  a host  comouter  end 
produce  code  for  e designated  sxternil  cohouter. 
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Cross*^ss*<nbltr  (003) 

A coffloutar  program  that  aceapts  symbolic  Instruction 
nnamonlcs  for  a salaetad  target  comoutar  ano  ganaratas 
target  comoutar  machine  coaa  «#>lla  hosted  on  another 
computer.  A cross-assembler  thus  allows  code  written 
for  one  computer  to  be  assamblao  on  another. 

Cross-raf eranca  program  (003) 

A group  of  cpnoutar  programs  that  orovlda  cross-rafaranca  Information 
on  system  components.  For  example,  programs  can  be 
cross-rafarancad  with  other  programs,  macros,  parameter  names,  etc. 

This  cepablllty  Is  useful  In  problem-solving  and  tasting  to  assess  Impact 
of  chennas  to  one  area  or  another. 

Cross-reference  tools  (007) 

Jtlllty  programs  *ailch  provide  cross-reference  data  concerning 
a program  written  In  a higher  level  language.  These  utility  programs 
analyze  a source  program  ana  provide  as  output  such  oata  as 
follows  I 

1.  Statement  label  cross-index 

2.  Oats  name  cross-index 

3.  Literal  usage  cross-index 

4.  Inter-subroutine  CALL  cross-index 

3.  Statistical  counts  of  statement  typos 
Cyclic  data  (Old) 

Oata  on  programming  activities  that  occurred  since  the  last 
management  reporting  period. 

Data  (Old) 

A set  of  facts  • 

Oata  analysis  tools  (012) 

r'rognms  specifically  designed  to  perform  statistical  and 
comparative  analyses  on  data  produced  during  the  execution 
of  a program  tost. 
h»t*  b*se  analyzer  (003) 

» co-wtar  progran  that  reports  Informntlon  on  every  usage  of 
dntn.  Identifies  each  nroeran  using  any  data  elements,  and  Indicates 
whethar  the  program  Inouts,  uses,  modifies,  or  outputs  the  data 
olanent.  \ny  un'jsea  data  is  printed.  Errors  dealing  with  misuse 
and  non-tss  of  data  and  conflicts  in  data  usage  are  Identified. 

Data  jsflnltlon  languane  (003) 

1 cnao-itar  urogram  uaed  to  dascrlbe  data  at  a sufficiently  hl^ 

Ijvel  In  order  to  mahe  the  use  of  a oartlcular  programing 
language  transoarent  to  the  Jeta  oaflnltion  process.  This 
langiiage  allov.'s  us  to  soeclfy  the  data  so  that  multiple 
larjuanes  can  share  and  use  It. 

Cata  Item  (On) 

A spsclflc  entity  of  data. 

Oata  reduction  tools  (012) 

Often  dats-base  dependent,  these  programs  orocess  a data  sat 
and  convert  the  Information  Into  rea-dable  form.  These  programs 
oerfom  statistical  and  comparative  transformations  on  the 
recorded  data  obtained  by  Instrumentation. 

, Debugging  (009) 

Oahugglng  starts  with  tnown  errors  and  attempts  corrections* 

Debugging  (012) 

* The  testing  of  a program  for  proper  execution.  Includes  the 

detection,  diagnosis,  recovery,  and  correction  of  program  errors. 

i 
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^Iso  Mt  VAlloation  and  Dabu'jgln?  Tools. 

Oobu2.7lng  tools  (012) 

l^ost  prograai  d«sltn«d  to  locatt  and  aliitlnate  prodranaln? 
trrort  and  to  tast  a program  for  propar  axacutlon.  Also 
aaa  Validation  ana  Oabudglnp  Tools. 

Dacltlen  tabla  (009) 

A tab'jlar  raarasantatlon  of  tha  following  tSraa  Itans' 

1.  Conoltlont  • factors  to  eonsloar  In  mating  a daelslon. 

2.  Actions  > staps  to  ba  tatan  when  a cartaln  combination  of 
conditions  exist. 

3.  Rulas  • specific  comp Inst  ions  of  conditions  and  tha  actions 
to  bt  tatan  under  those  conditions. 

-Oalivarablt  coda  Indicator  (019) 

Indicates  whether  or  not  tha  code  will  be  dallvared  to  tha 
cus tower. 

Delivery  (OOt) 

The  oolnt  at  which  tha  software  package  Is  turned  over  to  a customer 
for  use  In  the  operational  environment. 

Design  Inplenentatlon  specifications  (012) 

Ihls  document  defines  the  system  and  program  design.  It 
Includes  block  and  program  flow  diagrams,  set’wjseo  Inforaatlon 
on  all  data  descriptions  of  all  data  ano  may  Include  narrative 
descriptions  of  tha  operation  of  every  program  In  the 
system. 

Design  language  (003) 

A comojter  program  used  to  prelvJe  an  understandable 
r esresentstlon  of  the  software  design  as  It  evolves, 
ihese  orograns  allow  designs  to  be  constructed  and 
exoanJeo  In  a hierarchical  fashion.  They  oocument  the 
design  and  the  decisions  that  led  to  It. 

Design  requirements  baseline  document  (012) 

I'nls  oocument  is  the  baseline  description  of  tne  object  systemi 
hence,  the  foundation  upon  wh.lch  system  design  ano  conf Iguratlon 
control  orocaao. 

Design  simulation  (009) 

A lechnlPue  that  describes  e proposeo  system,  proouces  a computer 
nsscj  “mouel"  or  simulated  system,  ano  then  evsluates  the 
effect  of  various  system  requirements  anu  design  alternatives. 

Design  validation  (009) 

Tne  jranlnatlon  or  inspection  of  the  functional  ragulrenents  and 
tne  design  of  a software  system  for  the  ournos*  of  finding  errors, 
uther  terms  used  to  Describe  this  teehnlnje  or  variations  of  this 
tnchnlnof  incliioe  assign  review,  project  revle.,  ocslgn  Insoectlons, 
w2l  k-throughs . anc  Inororeas  reviews.  Tnls  technloue  is  similar  to  tha 
caslgh  verlflcctlen  technique  exccot  that  It  is  perfornej  earlier  In  the 
software  devclopmaht  cyclt  ano  at  a system  functional  level. 

Desl'fn  verification  (009) 

me  aranlnatlon  or  Insoectlon  of  a software  specification  for 

me  ourpose  of  finding  design  errors,  other  terms  used  in 

tne  literature  to  describe  this  technique  or  variations  of  this 

technique  Include  oeslgn  review,  ueslgn  inspection,  specification 

testing,  paper  testing  walk-through,  structured  walk-through,  nni  preliminary 


i 
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Design  ravltw. 

DavelPoment  test  and  evaluation  ((X)3) 

i'est  end  evaluation  that  focuses  on  the  technological  and  anglnearlng 
aaoects  of  the  ayitan,  or  aqulonent  iters  (AFa  60-14). 

DlsTtostlcs  oebu;  aids  (003) 

Compile  and  axacutlon  tine  checkout  and  debug  eapahllitlts 

that  help  Identify  ano  isolace  program  errors.  These  capehllltles 

usually  Include  commands  or  directives  such  as  D'JMP,  TAACE, 

•lODlrf,  CONTENTS.  BScAKPOiriT,  ate. 

Direct  interface  (OOl)  e | 

An  Interface  l<i>-»aoletely  between  two  software  elements.  : 

Ooeumentation  (012)  T 
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th*  proouctlon  of  fll  th*  aaptr  work  nwctssary  to  descrlb# 
tn  final  proouct.  Sxanolas  lnclud#«  cross-ref «r#nc#  listings, 
dictionary  listings,  and  flow  charts. 

-rtvjr  procran  (OOV) 

i loarfluo'js  (throw-away)  code  neeoed  to  perforn  the  unit 
tasting  and  lower  levels  of  Integration  testing  In  a hottoa) 
u?  software  oevelopnent  effort. 

Drivers  (DOV) 

Conouter  baseo  tnfornal,  testing  technique  and  are  useo  In  conjunction 
v/itn  other  techniques  alnost  exclusively  during  the  systen  Impl enentatlon 
onsse  of  s software  oevelopnent  project. 

D/nenlc  slnulator  (003) 

t conouter  progran  used  to  check  out  a prograia  In  a 

siTuiatso  environnent  slnllar  to  that  In  -.vhlch  It  will 

reside.  Closed-loop  effects  between  conputer  and  environmental 

noiels  are  gained  when  inouts  and  outputs  are  responded  to  by 

t-.e  various  mooels.  The  slnulator  allows  the  environment  to  be  stabllrsd 

at  a specific  configuration  for  any  nu 'ber  of  runs  required 

to  Observe,  diagnose,  and  resolve  oroblems  In  the  operational 

program. 

HJltor  (003) 

1 computer  progrsm  used  to  analyze  source  programs  for  coding 
errors  and  to  extract  Information  than  can  be  used  for 
cnacving  relationships  between  sections  of  code.  The 
a-ltor  -'111  scan  source  coue  and  detect  violations  to  specific 
oromramnlng  practices  and  stanaarcis,  construct  an 
ext»nslv»  cross-reference  list  of  all  labels,  vArlables,  and 
constants,  and  check  for  prescribed  program  formats, 

Egoless  progrannlng  (D17) 

this  term  was  coined  by  'lelmberg  (The  Psychology  of 
Co-guter  yrogramriing.  Van  '.'ostrand  Selnholo  Compeny 
:»■'  forV.,  1x71).  He  felt  that  some  arogremmers  can  become 
psychologically  ettschea  to  their  mrogreme  es 
extensions  of  themselves  so  thet  errors  Im  programs  become 
JT’aglm.g  to  the  orogrammer's  self-image.  An  egoless 
arogramnlng  environnent  la  one  that  craates  an  attitude  that 
ooen,  shared  progranmlng  Is  good.  By  making  code  publicly 
•vailahle,  the  ownership  of  cods  is  olscoureged,  and  individuals 
are  encsuregsd  to  write  coda  that  will  be  clear  and 
injeratandsble  to  others. 

clememt  ( 001 ) 

A Jrouolng  of  .Routines  which  performs  e perscrlbed  function. 

c-bejoeo  comojter  system  ECS  (010) 

•A  computer  system  thet  Is  integral  to  an  electromechanical 
system  such  as  a comhet  weapon  system  tactical  system;  aircraft. 

Ship  missile,  spacecraft,  certain  commend  and  control  systems) 
ano  clvliien  r/stems  such  as  am  lutomateo  rapid  transit  system, 
i-mheoueo  comouter  systsms  are  primarily  dlfferentlatad  from 
suto''etlc  data  processing  systems  (ADPs)  by  how  they  are 
jeveiooeo,  acquired  and  operated  In  a using  system. 

Its  key  attributes  arai 

1.  It  Is  ohysleally  Incorporated  Into  a larger  system  t^osa 
primary  function  is  not  data  processing. 

2.  It  Is  integral  to  such  a larger  system  from  a design, 
procurement  anu  oosratlona  viewpoint. 

3.  Its  outputs  generally  Include  tnfonutlon.  control 
signals  and  computer  data. 

Engineering  bclentlflc  Simulations  (003) 

• n engineering  emulation  is  used  to  study  system 
cnaractarlstics,  dsvalop  algorlthns,  and  provide  data  that 
act  as  a standard  far  tasting.  These  orograms  generally 
slnulata  subsystems  at  varying  degrees  of  complexity 
jeoenoing  on  the  subsystan  being  studied  or  the  use  made  of 
the  simulation.  They  generally  consist  of  a sat  of  nodules,  each 
of  which  Is  assigned  a soactflc  simulation  function  and  Is 
designed  with  wall-daflnad  inputs  and  outputs  and  precise 
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Intirftces.  Each  «o<Jult  ptrforni  an  astl^nto  slnulatton 
ftf\ction  to  vary  tso  nathod,  spaau  of  eomoutatton, 
accuracy,  cenplaxjty,  ate.  Tht  structure  can  aneonpats  all  basic 
alnulation  capobilitlat  for  sluulatlon  of  continuous  and 

discrata  aystams. 

Englnatrin’)  change  proposal  (012) 

/he  fornal  vahicle  by  which  a change  in  the  baseline  document 
Is  oroposad.  It  describes  the  nature  ana  nagnltuoe  of  a 
proposao  change  and  the  inpaet  of  the  change  on  all  elenents  of 
the  system. 

Environment  simulator  (003) 

A comojter  program  used  to  permit  testing  of  operational 
programs  on  a host  conputar.  The  operational  programs 
run  under  aimulatao  conditions  as  if  they  were  operating  within  the 
real-time  control  program  of  a machine  to  which  all  of  the  devices 
constituting  the  ultimate  system  are  attaches.  The  simulator 
program  contains  expansions  of  all  control  program  macros  that 
moolfy  the  entry  block  and  other  working  storage  in  the  some  nanner 
as  the  macros  in  the  actual  control  programs. 


Equate  program  (003) 

A eompjter  pregram  that  lists  all  equivalences  found  in 
exnmlnec*  code  ana  prints  warnings  when  multiple  equivalences 
are  founo. 

Est  mating  012) 

Oeternlnlng  what  levels  of  effort  and  what  resources  need 
to  be  amniieo  to  acconollsh  the  deslreo  results.  Also  see 
S'jdgetlng  ano  Estimating. 

Execution  analysis  (OOi)) 

I'he  eutornteu  monitoring  of  tha  computer  based  software  testing 
s'tlvltles,  collection  oata  from  these  tasting  activities, 
aivj  suhsenuantly  preiletlnn,  ty  menually  analytlng  the  Goto, 
the  ouratlon  eno  cost  of  tasting,  ano  the  ouallty  of  the  soff-vere 
oroo'jct.  Other  terms  useo  in  the  literature  referring  to  this  technique 
or  vsrlntions  of  this  technique  Incluoe  code  analyzer,  cooe  auoltor, 
program  evaluator,  ano  proouct  assurance  evaluator, 
rllght  tests  (003) 

» technl.-ue  used  to  oemonstrate  haroware  and  software 
oarfor'ence  in  actual  system  ootratlon. 
r'low  cherts  (012) 

diagrams  of  a program's  logic  flow, 
r lOKChart  (OOT) 

A gramhlcel  represenintlon  for  the  definition,  analysis,  or 
solution  of  3 orooltm,  in  v/hlch  spools  ere  usao  to 
reeresant  operations,  oata,  flow,  equioment,  ate.  (A*lSI). 
r lov.-emarter  (003) 

‘ co-'P'jter  program  used  to  show  m oetell  the  logical 
•tructure  of  a conouter  orogran.  The  flow  is  Determined 
fro-  the  actual  operations  as  specified  by  the 
executePle  statements,  not  from  comments.  The  fiowemarts 
oenerated  can  be  eomoored  to  flowcharts  provldeo  in  the 
co-outer  program  soielf leetion  to  show  dlscrepanclas 
ahj  illuminate  differences. 

Flowcharting  tools  (007) 

Jtlllty  programs  which  atuometleally  draw  program  flowcharts 
directly  from  source  code. 

Formal  Validation  (Oil) 

'lethemstlcel  techniquas  for  proving  program  correctness. 

Forms  1 testing  (001) 

resting  conducted  according  to  test  procedures  which  are 
documented  and  approvtd  by  contractor  and  customer, 
rormel  testing  (009) 

Zesting  ptrforrao  in  accordance  with  customer-oporoved  test 
plans.  This  type  of  testing  verifies  that  tha  software  system 
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Is  oosratlnij  accoraing  to  th«  r»<iul rsnants  of  th«  tievelopnant 
soscl  fl  c.itl  ons.  ror"'.ii  tastln-j  Is  us'jally  oerforntao  -Jurln-j 
t:*^  systsTi  av  a I u."' 1 1 on  .ohssa  of  soft’-'sr*  Jsveloomsnt.  i>rns 
•jsau  In  t'la  lltsrstur*  to  aescrl'oe  tMs  tsstlni  IncluOai 

1 oyste-a  IntS’jrstlon  Vsstin:] 

2 t^rototyoa  Tasting 
J S/st«“  Testing 

Arcsptsnc*  fasting 
r'jnctlon  (001) 

A grouping  of  routines  which  performs  a prescribed  function, 
runct Ion  (012) 

A S'jn-ilvlslon  of  processes, 
p'jnctlonsl  regulr events  (OOM 

Describe  the  functions  the  system  aiust  perforn. 

.-•inctlonel  specifications  (012) 

Describe  a syste"  In  terns  of  Its  principle  functions  and  their 
interrelationships,  l.e..  the  functional  relationships  of 
tna  oarts. 

runctlonal  testing  (009) 

jne  execution  of  Independent  tests  assigned  to  cenonstrate 
a specific  functional  caoablllty  of  a nrogren  or  a software  systen. 
De.nerator  (OOJ) 

A generator  produces  test  data  or  test  cnses  to  exercise 
tne  target  systn*'.  A generator  In  this  case  Is  differentiated 
fron  a sinulotor  because  It  actually  creates  test  data 
using  .nu"'erlcnl  Intenrators,  ranJo-"  nunber  generators,  etc. 
unce  the  oata  are  produced  ov  the  generator,  a simulator 
■light  he  required  to  route  the  oata  to  the.  systen. 


Jenerators  are  useful  In  a systen  test  environnent  where 
•’live"  oata  Is  not  aval labl e.  useful  output  of  a data 
generator  are  t-npes  of  logged  data  that  can  be  used  with  a 
j.ita  reclay  facility  for  estacllshlng  standard  test  cases. 

UPt)  (00-1)  . 

.(lerarchy  plus  Input/ PrPC ess/output  Is  a graphic  design 
lacnnlgue  useo  to  shov?  function.  !•!?:)  alagrans  describe  functions 
In  term's  Pf  the  Input  to  a process.  They  show  a syste's, 

S'jnsyste'T,  or  prngren  f unctionally,  l.a,,  the  functions 

th’t  It  cerforrs.  ans'-erlnp  the  question  "/(hat  does  It  do?" 

ol.nes  these  Jlanra.ns  are  visual,  they  are  easier  to  understand  than  host 

dPcunentatlon  which  is  r.arratl-/e.  Although  flp-./charts  are  another 

grephlc  design  technique,  they  show  ornanir.atlpn  ana  logic  In  contrast 

to  function. 

herd /ere  nonltors  (003) 

A "nit  that  obtains  signals  fro"  a host  co-iouter  systen 
through  probes  attached  directly  to  the  cP'aputer's 
circuitry.  The  signals  obtained  are  fed  to  counters  and 
tl'ers  ano  are  recorded.  These  data  ara  reduced  to 
octaln  Infornatlon  aoout  CPJ  utilization,  channel 
activity,  etc.  These  -data  can  be  ussd  to  Improve  both  system 
eng  nrogra's  perfomance. 

Inolp.rentstlon  nlslntarpretatlon  error  (013) 

An  error  for  a unit  of  source  coot  assoclateo  with  a program 
due  to  the  nlnlntarorocatlon  of  the  progran  specifications. 

Iiolleu  systei!  parforiaanca  (002) 

An  ijnwrlttsn  requlranant  which  Is  understood  by  the  majority  of  tha 
■>ro  )ect  tea.'!)  to  he  essentially  equivalent  to  a wrlttan  requirement. 

Inforngl  proof  of  correctness  (012) 

The  visual  inspactlpn  of  a snail,  conprahensl va  sat  of  test 
cases  inolcatlng  that  the  cod#  of  a progran  segment  mttches 
Its  speclflcetlon.  Validation  of  tha  oro.gram  sagnant  Is  based 
on  axioms  stating  that  lower-level  segments  match  their  specifications. 

Infor'aal  testing  (009) 

Ttatlng  that  utilizes  Internal  test  documentation  control  and 

, ... 
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oroctd'jrts.  InforTal  testing  ufuelly  Is  ocslT^eJ  to  h« 
dtvtJopmsnt  ^roup  ttstlnp  anJ  rtouirts  no  fornil  customer 
aoprovsl.  Informal  testlnT  usually  bfjlns  when  the  first 
procrem  unit  Is  eoPea  and  continues  throu7hout  the  system 
Inolementntlon  ohnse  of  software  oeveloonent.  Terns  used  In 
the  literature  to  describe  this  testing  Include' 

1 Unit  Testing 

2 S'lPsysten  Testing 

3 Integration  Testing 

4 Component  Testing 

5 Oevelooment  testing 
Infor-'stlon  (013) 

Correlation  of  data  for  the  process  of  Infornlng. 

Instruction  simulator  003) 

4 com>jter  program  used  to  simulate  the  execution 
characteristics  of  a target  computer  using  a sequence  of 
l.mstructl ons  of  e host  comouter.  The  Instruction  simulator 
orovides  blt-for-blt  fidelity  with  the  results  that  would 
be  produced  by  the  target  conpijter  following  the  same  operations 
ana  Initial  conditions. 

Instruction  trace  (003) 

A compjter  program  used  to  record  every  tine  a certain  class  of 
poeretlons  occurs  end  trigger  event-driven  data  collection. 

In  seme  esses,  this  creates  a comolete  timed  record  of  literally 
everytning  significant  thet  occurred  during  program  execution. 

These  traces  contain  data  on  instruction  and  become  a 
permanent  record  of  a progrem's  execution. 

Instructions  (ODD 

iachlne  instructions  (machine  deoendent  parameter) 

Instrumentation  tools  (012) 

inose  procrems  that  monitor  enc  record  Inforratlon  about  an 
Object  system,  or  oortlons  thereof,  es  It  oocraies  oats  reduction 
enu  analysis. 

Interface  chec'cer  (003) 

\ commuter  prorrn”  used  to  automatically  chee):  tne  range  and 
ll'lts  of  varlehles  es  v;elJ  as  the  seeling  of  source  orograms 
to  assure  format  connllanca  ’••■Ith  interface  end  control  documents. 

Interface  soecl fi catl on  jocu’ent  (012) 

•\  joc'jnent  tnat  serves  ea  • eo“.muml  cetlons  vehicle  between  the 
Oo'.f  1 gure ;i  on  Control  anc  Technical  Inolenantatlon  processes, 
sns  sunports  tne  coordination  of  efficient,  controllable 
Interfaces. 

Internal  osllvery  (001) 

ine  bolr.t  at  vhlch  tne  sofware  as  en  entire  pse)cage  Is  given 
to  the  Inuepenrent  test  qroun. 

Interrupt  analyser  (033) 

' co-aiter  program  in*t  eyanlres  source  code  and  determines 
botentlel  conflicts  In  the  use  of  oats  snJ/or  storsge  due  to 
1 nterrjbts. 

JD/li.i.  (012) 

*.  progranning  language  oeslgned  soeclflcally  for  command  and 
control  projects. 

Job  (013) 

Computer  Job  consisting  of  one  or  more  steos  such  as  conollstlon, 
assanbly,  or  utility  runs. 

Lanqjsge  processors  (003) 

Computer  programs  useo  to  translate  high-level  or  symbolic 
instr'jctl on  mnemonics  Into  eomputer-orl entao  cooe  ceoable  of  being 
oseyeo  by  a co-outer.  These  nroe«ssors  typically  heve 
csoablllties  for  error  Detection  through  syntax  enalysls  ano  arovloe 
sv'.hollc  addressing,  expression  evsluatlon,  and  symbol  cross-reference 
listings.  Compilers,  assemplers  and  neta-asscmolars  are  axamoles 
of  this  category  of  aids. 

Levels  Of  abstraction  (012)  i 

4 design  ano  ir.pl  ementeti on  method  that  heloi  proouee 

relladle  software  thet  Is  rore  easily  modified  ano  malntelneo  ' 
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^3/  13*ntl*yln5  ^no  placing  into  a niararcnical  stpjctura  tha 
functional  procassas  ana  oitj  resourcas  tnat  constltuta  tha 
syst'JT's  pro'jra-"  jtrictura.  In  aurlitloo,  a atitl-jn  dlsclpllna  that 
s’jooarts  tha  creation  of  a «all  hahava^,  hi ararchlcally 
struct'ireJ,  nooular  systa. 

Lin*  of  so'irce  coda  (013) 

■■3  ch*ractar  card  Inaj*  of  sourca  cooa. 

Linas  of  sourca  cooa  fron  anothar  sourca  (OM) 

Cooa  not  davalopao  Put  artrnctad  fron  othar  sourcas'.' 

LoaJnule  pro.jran  a<!tn  (Q12) 

0*ta  that  Is  ralocntabla  or  atsoluta  binary  nioJules  producad 
by  3 llnic  edl  tor. 

Lb* oar  (003) 

» conpjtar  oronrnn  that  anables  axtarnal  rafarancas  of  symbols 
inon-r  diffarsnt  assanijlias  *s  ■•■/all  as  tha  assl-jmant  of  absoluta 
aourasses  to  ralocatsola  strln-js  of  coda.  This  nrogran 

provijas  ola.-jnostlcs  on  assancly  overlap,  unsatlsflao  external  references, 
*nj  .-;ultl.)le  oaflnaJ  external  synbols. 

Lo'l:  jc'jatlon  generator  (003) 

\ cam-Jtsr  pro.;ran  used  to  autonntl ca  11/  reconstruct  arlthnatlc  text 
ena  tb  flo./chart  ssser.hly  l-*n-;'ja;ja  pronrens.  One  s'jch  oro-jraai 
trenslites  essanblv  lanj'iane  Instructions  Into  a nachlne-lnJepenoent 
nlcrobro-;r-emin-;  lannuaja  and  ljullds  tha  nl croorO'.;ra.-;nln3 
statenents  Into  a nat’oor't  In  which  flo.-i  of  control  Is  analyzed 
anj  euatlons  reconstructed. 

'.\r>  orocran  (OOJ) 

\ coho-jter  oro^nn  used  to  orovlJe  locntlon  and/or  size 
infornatlon  anout  nil  or  selected  oarts  of  tha  target 
s/st*n,  or  aaoiit  devlca-resloent  data, 

'ajor  error  (002) 

A catastrbbhlc  event  ■■••■hlch  Intarruots  or  could  l.nterruat  most 

of  *11  na  Jor  s/Sts'*  functions,  a.'j.  an  Injlnlt*  loop,  system  crash,  a 

major  memory  overflow,  a ■oate  'oase  corruption,  etc. 

'ina;*mant  (012) 

A tern  that  Inulcatas  nethodolopy,  tools,  and  orocaduras. 

’■an33e-"»nt  control  *nd  orolect  visibility  (012) 

Ihos*  processes  tmt  monitor  the  oroject's  status  In  respect 
to  yl  inned  Isval.s  of  scnaoula,  cost,  and  performance,  an-J  take 
corrective  action  If  necessary.. 

' anage'cnt  functions  (005) 

Alt'iO'ugh  tha  uenacencnt  Process  has  been  Jescrlbeo  in  many  ways, 
f>ir  oasle  functions  have  recelvad  general  acceptance  - planning, 
dr';anlzlng,  controlling  end  ronmunl ce  tl  ng, 

1.  f/laaning.  I'he  f'jnctlon  to  determinln)  the  project 
opjectlves  ena  tha  policies,  orograms,  procedures  ana  nethods  for 
acnlavlng  them.  The  planning  function  must  provide  a framework  for 
decision  making. 

2.  organizing.  The  function  of  determlng  the  activities 
re-'jlreu  to  schlevs  ths  objectives  of  e orogramming  project,  tha 
Jeoartmentatlon  of  these  nctlvltlas  end  the  esslgnment  of 
authority  and  responsibility  for  their  oerfornance. 

3.  Control.  The  function  of  assuring  that  the  various  co.mponents 
of  a orolect  are  performing  in  accordance  -with  the  olan.  Control 
Is  essentially  tha  measurement  and  modification  (if  necessary) 

df  comdonent  activities  to  assure  the  acconpl  l3)vmsnt  of  ths 
oversll  plarf. 

4.  Communications.  The  function  of  transferring  Information 
anong  decision  makars  throu^.out  tha  project. 

.snoganent  statistical  data  (013) 

laneral  name  appllsd  to  all  the  data  collected  end 
accumulated  by  tha  .OSL  for  tha  .ouroosa  of  prod-jclng  management 
rcborts  Including  hoth  Plan  and  Actual  data, 
v.snagemtnt  statistical  data  base  (010) 

A data  base  containing  Menagsnent  Statistical  data  for  an 
ongoing  programming  project, 

’'snual  basau  tasting  (009) 
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1^  Testing  tMt  Is  usually  dlractad  at  avalustlnn  both  the  design 

ano  the  product  <l.e.,  programs  anJ  documentation  ).  The 
oestgn  Is  usually  evaluated  from  documents  containing  Information 
such  as  functional  requirements,  system  specifications,  and 
program  specifications.  The  product  avaluetion  usually  Involves 
review  of  the  computer  programs  and  the  documentation  describing 
the  programs  or  systems. 

Patacomoller  (012) 

A comoller  system  designed  specifically  to  Implement  (compile) 
language  compilers. 

Metric  (001) 

A ntes'ire  of  the  extent  or  degree  to  *rilch  the  softr.'sre  possesses 
ano  exhibits  a certain  characteristic,  quality,  property,  or  attribute. 
Vlnor  error  (002) 

A marginal  event  «f^tch  allows  or  could  allow  some  portions  of  the 
system  to  operate  properly  while  Interrupting  others,  e.g.  some 
missing  output,  son-e  wrong  output,  an  Inaccurate  computation, 
e racovtrable  transient  error,  etc. 

Mlsts'se  (Oil) 

A human  action  producing  an  unintended  result. 

Modeling  and  simulation  tools  (012) 

iools  used  for  trede-off  studies  and  to  Investlgste  psrtlculai 
abstractions  and  aaproeches  for  the  system  oesign.  They  are 
useful  for  analyzing  and  moarllng  particular  approaches  to 
system  oeslgns.  Examples  includeiCASE,  CSS,  C?5S,  UOOIIT,  SCE^T, 
and  5PCL. 

1.0-Jlf  lability  (013) 

I'clles  control leo  change.  In  which  some  parts  or  asoccts 
remain  tht  sane  while  others  arc  altered,  all  In  such  a 

way  that  a oeslred  new  result  is  obtralned. 

’•■Piular  pro  jramr.i  ng  (003) 

tnr  technique  of  producing  relatively  •"'all,  easily  interchangeable 
computer  routines  v:hlcn  meet  certcln  stanaarojaed  Interface 
r e'jul rements.  This  technique  me'<es  It  easier  to  oevcloo  and 
verify  completed  computer  oronrans.  '-'oduleri  ty  is  acconplisned 
b/  br?ai:lng  the  program  into  llmlteo  11  ne-semmant s that 
aerfor*  complete  function*  anc  are  tnerefore  conpletely  unuerstandatl e 
in  themselves.  Alas  that  help  Implement  these  techniques  ere 
standards  an-  procedures. 

Vcj';: ’-.rlty  (013) 

deals  •••1th  hP''  the  structure  of  an  object  can  make  the 
nttalmeni  of  some  purpose  easier,  '(pauisrlty  Is 
•5jrpoa»ful  structuring. 

VPdul*  (Ol*-) 

*.  nro‘Tr''m  unit  that  Is  discrete  and  Identifiable  with 
reSp-JCt  to  conpiJing,  contlnlng  with  other  units,  and  loading. 

()bl*ct  program  oate  (012) 

,n»  resulting  for"  of  a source  Jenguegc  program  after  processing 
jy  r corpllar  or  assembler.  They  are  also  celled  object  Vooules. 

fhc  oblect  program  Is  in  a format  suitable  for  loading  ano 
execution.  It  nay  require  auoitlonal  processing  by  n link  loader 
or  link  adltor. 

Ooeratlcnal  OOl ) 

/n«  status  4iven  a software  package  once  it  has  completsp  contractor 
testing  and  its  turnad  over  to  the  avantual  user  for  use  in  the 
anlicetlons  envlronnant. 
overlay  program  (003) 

com>jttr  program  that  allows  specific  system  components  (load 
modules,  core,  data  base,  ate.)  to  be  moclflcd  curing  cxccutlo*! 

In  the  case  of  mocrjles.  a orogra^e  vith  an  arror  can  be  replaced  in 
core  without  bringing  the  syste*  oown  anp  starting  It  up  again, 
dystem  parameters  tnat  effect  oerforaence  can  be  varied 
d'jrlng  execution  to  compare  vaarlous  priority,  timing,  etc., 
schemes. 
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P5L  joo  (018) 

*11  of  th*  co.Tiputflr  orocwsslno  that  results  from  a alnijl* 
ij3»r  reouest  to  execute  the  PSiL  for  the  ouroose  of  parfornlnq' 
one  or  more  P3L  functions  such  as  uP'Oete,  conolle  or 
output. 

Path  analysis  (007) 

\ soft'ere  technique  'Ahlch  scans  source  code  In  order  to  design 
on  optional  set  of  test  cases  to  exercise  tne  orlnary  paths  In  a soft-ears 
“10  (ul  e. 

Performance  (OOV) 

ins  evaluation  of  nonloglcal  properties  (l.e.,  computer  run  tine, 

resource  utilization)  of  a software  system.  Performance  Is  measured  In 
terms  of  the  amount  Of  resources  required  by  a soft-wars  systen  to  product 


0 result. 

Per  form.omca  oriented  oats  (012) 

J-oformetlon  reqsroln;  mamaqement  Items. 

.-er r'or^amce  requl ra-oents  (006) 

jjecify  the  tl-^e  and  soace  constraints  which  must  be  met. 

Perlymeral  slnul-itor  (OOJ) 

' co--3utar  program  used  to  test  crltlcel  co.mputer/qer lohera I 
Interfaces  that  exist  In  real-time  aopl Icat Ions . these 
simulators  range  fro*'  functional,  in  -iTiich  case  the  perlohoral 
orovUes  feeo'.rac':  to  the  pro-;ram  on  the  assunqtlon  that  all  Interface 
constraints  are  satlsfleo,  to  h lgh-?l aell ty  slnuiatlons  In  -.-(hlch 
I'le  Interface  constraints  -ust  be  modeleo  to  a detailed  level 
of  tl-'lno  ar.s  message  for“ettlng. 

.“ian  )-»t3  (03) 

)ata  describing  ths  method  or  scheme  of  action  for  a project 
that  -will  t«  included  in  the  evanegement  reports. 

Planning  (C12> 

•i  technique  that  Incl-jdes  the  evaluation  of  project  requlrenents 
in  tarn's  suen  that  logical  asslgnm.ants  can  be  mads.  Also  sea 
.-’lan.mln,-  and  dch*uullng. 

Plan-il'g  eno  schecullng  {012) 

All  activities  that  Incluoa  an  evaluation  of  the  project's 
requirements  end  making  assignments  to  conduct  the  project, 
rre-evecutl on  tools  (012) 

.'ools  that  opereta  on  the  linguistic  description  of  a program 
»nc  oo  not  roaulre  Its  execution.  Examples  Include*  syntax 
ciecblng.  Interactive  conpil«rs,  progra-s  reference  listings, 
flow  charts,  an-d  re  formatters, 
rreco  -.pller  (00/) 

\ particular  type  of  com.iuter  program  -vhlch  has  ths  following 
chsrecterlstl cs » 

1.  It  Is  normally  executed  Immediately  preceding  a program 
comp! la  tlon. 

2.  Its  Imput  consists  of  orovcrammlng  statements,  of  which  all 
or  part  sra  unaccaotabla  to  the  compiler. 

.1.  It  generstss,  as  output,  a comouter  program  In  a syntax 
accept.abla  to  the  compiler. 

Process  construction  (003) 

A tachnlqua-usad  to  combine  and  link  Inuapsndently-codad  modules 
into  a run-tine  process.  These  Include  llnksnes  to  the  operating 
systen.  The  technique  allo-<s  for  rapid  reconfiguration  based 
j on  stimuli  from  the  run-tlna  environment  of  a software  systen 

I to  reflect  changes  made  to  a number  of  Its  modules.  Specific 

comoutar  programs  are  available  that  serve  as  sUs  to  1 mo  lament  at  Ion 

I These  Include  sped  si -pur  pose  editors  and  control  orogra-ns. 

?roj*jct  (012) 

everything  contractea  for,  oroducad  for,  -and  delivered  to  the 
custonar.  Examples  Include*  hardware,  programs,  documents, 
and  training. 
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(■roo'jct  cvrtlfl ccstlon  (012) 

A o^nonstratlon  th«l  t^e  vctuol  performince  eorrtsponds  to 

txptctoo  iystn  oerfomanc*. 

Proa-jct  i»irr«nty  (012) 

A proctss  that  oractots  conf l7jratl on  control  and  quality  aisuranca 
activities.  This  proctss  Is  a device  for  eustnnsr  feedback  after  a 
software  systen  has  been  oellvereo.  It  provioes*  (Da  aeons 
by  v^lch  the  customer  reports  susoected  problensj  (2)  a aaans 
for  technical  and  contractual  evaluation  of  thase  probleasi 
(3)  arbitration  and  apoeal  procedures!  and  (4)  a means  to 
re-certl fy  the  correcteo  systen.  Also  see  Product  I’.oiTanty  and 
'lalntenance,  ana  Product  vtarranty  Procedures. 

ProcTjct  warrcnty  and  maintenance  (012) 

K uevice.  Indicating  the  procedures  to  be  followed  during  the 
warranty  period  of  a system,  for  customer  feedback  after  the 
delivery  of  a system. 

Proouct  warranty  procedures  (012) 

A customer  feedback  device  that  precades  the  delivery  of  a product, 

anj  Indicates  the  procedures  to  be  followed  during  the  warranty 
period  of  a system. 

!-rodjcilon  llororles  (003) 

A technique  used  to  orovlda  constantly  up-to-date  representations 
of  the  comouter  programs  and  test  data  In  both  computer  and 
human  reodihle  forms.  The  current  status  and  past  history  of  ell 
code  genereteo  Is  also  melntalned.  Specific  library  programs  era 
ovmileble  to  serve  as  aids  to  Inolenentatlon. 
rrod-jctlon  run  (012) 

The  operotlon  of  a software  systen  under  real  operating  conditions 
and  the  production  of  useful  products  for  the  customer.  This  Is 
cp'itraslea  with  a test  run,  which  is  the  ooeratlon  of  a software 
system  to  test  Its  oerfornsnce. 

Prooucilon  tools  (012) 

Tools  related  to  the  specific  requirements  for  software  system 
pro'jiictlo.n.  Examples  include!  Structures  Programming  Design 
Tool,  Program  Production  Library,  Program  Validation  Tools, 

Program  atiih  Simulators,  compilers,  link  editors,  and  source 
orogram  editors.  See  Program  Proouctlon  Tools. 

Prouuctivlty  (OOP) 

iridl tlonelly,  the  generally  acccoted  description  (If  not 
Jiflnltlon)  of  progra-mlng  productivity  has  teen  "llnes-of-cooe/ 
men-Tontn"  (l.e.,  quantity  of  code  oroducea  . Contemporary  research 
sugrests  that  the  definition  of  oroouctlvlty  should  somehow 
contain  st  least  three  additional  elements* 

1.  A quail tatlv*  element  concerned  with  the  correctness  and 
efficiency  of  a program, 

2.  * qualitative  alemant  eoncemtd  with  complexity  (l.a.,  the 
difficulty  of  eommrtnenclnr  the  apolleatlon  (or  algorithm) 
celng  Imlementeo  and  the  orgcmliatlon  of  the  orooram  hlcrarchlal 
tree  structure  (e.g.,  number  of  levels,  width,  oeta  Interfaces  ). 

3.  An  element  eoncarnec  with  the  cost  of  the  program. 

Program  (015) 

The  lowest  laval  of  module  that  can  be  assembled  or 
cs-iplled  and  can  be  executed  at  a single  entity, 
i-rogrem  :)fflce  (003) 

The  flalu  office  organised  by  the  Program  ‘^anagar  to  assist  him 
In  accomplishing  the  program  tasks  (APP  900-2). 

Program  oescrlPtlon  specifications  (012) 

These  are  informetlonal  Oocumants  produced  for  the  customer, 
usually  according  to  Ms  raqulrements. 

Program  ueslgn  language  (005) 

(POL)  a oaslgn  tool  usao  to  facilitate  ths  translation  of 
functional  specifications  into  eonputtr  instructions. 

Program  oavelopoant  tools  (012) 
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I'ools  tak»  ih*  coip*it»r-stor«3  sourc*  pro-jran  Into  an  object 

coje  Tootila,  than  ^ loab  no.iiila,  ano  subsaTuently  execute  the 
coje  n a specific  cast  anvlroment. 
r'rojra-'  flo-  analyzer  (00.1) 

1 eoi.aijtsr  pro^ran  that  orovltias  statistics  on  sourcs  coda 
ifteMant  usage  and  lining  Jeta  on  progrsn  elensnta  during 
test  cise  executions, 
r-rogran  1 '.ole-’entat  Ion  (012) 

‘11  ectlvltles  associated  with  software  nodule  oeslTif  coding 
ano  testing  ano  the  Integration  of  software  nodules  Into  a 
functional  systen  operating  on  the  total,  final  hardware 
con  figuration. 

.srcorac  Instrunentatlon  (012) 

A cji.ntltatlve  assess-ent  of  how  thoroughly  a orograrj  is 
exercised  by  a set  of  test  cases, 
rrogr*-  Instrunentetlo.n  tools  (.012) 

ionls  that  orovlae  a ■'echanlsn  for  nonltorlng  and  recording 
1 .nr nrnatl on  about  an  oojact  progran  as  It  ooerates.  Sxanples 
include  trices  cnu  snapshot  Junps. 

.cro-ren  lane-enent  'Jircctlve  (003) 

I'he  offlcl.al  i*"  L’3Ar  ranansnant  directive  used  to  provide 
direction  to  the  Inpl ene.n ting  ana  partlcl  dating  conTa.nos  and 
satisfy  doc'jc'entailo.n  renul ranents.  It  will  be  usee  during  the 
entire  acoiilsltlon  cycle  to  stata  ragul  re na.nts  and  reguast 
stunles  15  well  es  Initiate,  aiorove,  charja,  trsnsltlon, 
iddlfy,  or  terminate  procrans.  fha  content  of  the  P.O,  Including 
t li  ragulrsc  iiO  UiAr  review  en.l  .approval  actions,  la  tailored 
to  the  needs  of  each  l.ndl visual  progra.n  (ArH  100-2). 

.-'rograi  .lancganent  pl.an  (003) 

ihe  docinant  oavolopad  and  Isaued  by  the  .^rografl  .Manager  that  shov/s  the 
l.nte:ratej  tl.Ta-phasec!  tas'.rs  a.nd  resources  rcautreo  to  conplete  the 
tas':  sooclflau  In  tha  P'O.  i’ha  P"0  Is  tailored  to  the  naoJs  of 
3'r'  Inulvidu.al  pro.gran  (API  t;0O-2). 

'-rogra  nana.gar  (003) 

i'l’  :anorlc  tern  usao  to  danote  a single  Air  rorce  .Manager 
(J./ste-  !'ro.;ran  Mlractor,  ?rogran/?ro Jact  lanager,  or  Systen/Iteis 
'rn.agar)  during  any  sosclflc  phase  of  the  acquisition  life  cycle 
(A.-!  100-2). 

rrogre-  nrouuctlon  t.nol  (012) 

An  eacadllshac  .nrocaJure  or  conoiitar  progran  that  provides 
assistance  In  t.he  devaloonent,  Inpla.nantatlon,  and  testing 
of  a soff-ars  systen. 

'rmran  raferancas  (012) 

idaclal  listings,  nroducad  hy  nany  connllars,  that  Increase  the 
user's  uncarstand Ing  of. the  nature  of  the  orogran  nroducad. 

2ri  -alas  lncliwe»  dictionary  listings,  cora-storige  naps,  and 
crass-raf are.nce  listings, 
f'rogria  ssgr.ar.t  (012) 

A connin.atlon  of  nragran  stans  and  calls  to  lower-level 
drograe  segnents. 
rrogra.'  saguenctr  (003) 

A co-’->.jtar  progran  used  to  force  execution  of  all  posslbla  . 
nrogran  instructions  ana  branches  to  deternlne  progran  flow,  exacuto 
saluon-ustd  oranchas,  and  to  verify  proosr  progran  operations. 

.ne  ltd  Is  often  used  with  an  Instruction  slnulator, 

I'rngran  stub  (018) 

A t an jorary  (l,a, , dumy)  unit  of  source  coda  which 
Is  part  of  an  ineonpleta  structured  progran  an.d  will  be 
replacad  by  tha  actual  unit  of  coda  whan  It  Is  conplatad. 

Progra-  stub  slnulatsrt  (012) 

3«nerallzad  subroutines  used  In  .orogran  Stubs  that  supply 
the  coda  racuirad  to  a.atahllsh  tha  desired  lln'taga  with  tha 
hlgnar-lavei  progran  sagnents.  They  Incluae  ganerallzad  table 
looVuo  routines,  randon  nunoer  generators,  or  routines  that  only 
record  tha  Invocation  of  a progran  sagrawat. 

Progran  stubs  (012) 
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r>«nnv  program  st^ntnts  eontalnln;  anou^h  coPc  to  actabllsh 
llnko^t  with  a hl^tr>lavtl  aa^mant. 

►roTran  valloation  (012) 

All  tachnlmas  usafl  to  ahiura  corract  pro7ramt.  Including  aystam 
and  suh-systan  tastt  and  sytta'i  Intasrstlon  tasting, 
frorira”'  valloation  tools  (0)2) 

fools  that  ara  usad  in  tht  praaxacjti on  and  run  tl»a  phasas 
or  i>ro7ren  Inpl avantatlon  and  Proyr»'»  Validation. 

ProTranr.inT  support  lisrery  (005) 

(rot.)  a raposltory  tor  osta  naeasssry  for  tha  erdarly 
Javfl opuant  of  co-outar  projrans  using  strueturaJ 
preyrannlng  tachnology.  iha  data  raposltory  Is  In  two  fornsi 
data  Is  storad  In  nachlna  raaoabla  forn  accasslbla  by  tha 
conp'jtar  ana  tn#  Identical  data  Is  storao  In  hard  copy 
farn  in  project  notaooolct.  A ?SL  also  Includes  the 
necessary  eonoutar  and  office  procedures  for  nanlpulatinp 
this  data. 

Project  (DI2> 

•.11  pracassas  necessary  to  produce  a product. 

Project  (TIS) 

A paneral  tarr  used  to  describe  a software  davelopaient  effort. 

Project  construct  data  (012) 

Information  on  design  end  laiolanantatlon  details. 

Project  uote  base  (012) 

A project-specific  catalog  containing  nanaganent  data  ana  program 
date  S'jppll«-i  and  useo  by  various  facilities.  Exenples  include)  the 
contents,  nenrs,  anu  llnrspes  of  all  tha  nojulas  In  a oartlcular 
s/sten.  j'Se  ranagenant  data  pertelnln;  to  the  tasks  and 
nllestones  of  a specific  project. 

Proof  of  correctness  (DI2) 

/Tool  that  a oronren  produces  correct  results  for  all  possible 
inuuts.  Vallcetion  of  a program  In  the  sena  way  a irsthanatlcel 
theora-.  Is  provao  correct,  l.a.,  oy  netnanaiieai  analysis  of  Its 
orooartlf s. 

Proofs  of  correctness  (017) 

An  clt"rnatlve  to  axecjtlnn  tests  Of  software  to  danonstrata 
Its  correctness  Is  me  nethod  of  enelytlc  oreofs.  The 
verlf lection  nrocess  con.alsts  of  n».;lne  assertions  Describing 
the  .state  of  • orogra"  ihitlelly.  at  intarheclate  points 
ih  tne  orooren  floi,  ano  ni  ternlnetl oh,  ahd  thah 
orovih-i  that  enrh  essertlon  Is  Imlleo  by  the  Initial  or 
prior  one  ani  th»  trnnsfomstlons  oarfornao  by  the 

erogrpn  oetwean  each  two  consecutive  assartlons.  An 

a.asertlon  cnn.aist£  of  a definition  of  tne  relotlonihlns 

snoni  the  vsrlnhles  rt  that  point  In  the  orograT  where  tha 

'sscftlon  is  nase.  i'he  oroofs  snplo”  stanjard  tachnlpuas  for  proving 

tnaorans  In  the  first  orcer  predicate  calculus.  Proof  of  the 

correctness  of  a program  using  this  scoroom  Deviates  tha 

mate  for  avac’jtlng  test  casts,  since  all  possibilities 

nre  covaroo  by  the  proofs. 

S'jolity  RS.auranca  (012) 

.‘ne  orocas!  of  activity  during  which  tha  system  ueslgn  Is  auoltad 
tc  ciStarr.Ina  eciathar  or  not  it  raprasants  a verifiable  ano 
certlflnbla  soacl fl cation,  and  ourtng  which  tast  plans  ano 
last  orocaduras  are  forDjlatao  nne  imnlamanted.  This  activity 
ensures  tha  technical  eempl lance  of  tha  software  system— a 
orou-jct— to  Its  Rapulrahants  anu  Otslmn  sped fleatlons.  duality 
Assurance  It  an  Inotpandtnt  audit  review  of  all  products  to 

ensure  their  eonplianet  to  a nanagamant-dlrecttd  stanoerd  of  quality. 
Raeoro  generator  (003) 

t comj-.'tar  program  used  to  construct  tast  data.  Essentially, 
tha  orogran  contains  a library  of  date  fomats,  including 
tha  location,  else,  character  (alpha  or  numeric),  ano  normal 
sontanti  of  each  field  of  each  racord  type  fron  which  It  ganerstas 
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rsroraj  r“'qulr«iJ  for  testing, 
i.soortlng  (012) 

• ’sjor  SMb-Ol  V Is  Ion  wtt^iln  Oonf  1 inratlon  Control.  The 
reoorttnj  snu  cocj'^entlnT  ^ctivites  neejeJ  to  lonltor  the  status 
of  conf Ivuratlon  ujrlnj  the  Ilfs  of  a s/stan. 

11  egijl  reients  anelysls  (012) 

.he  orocess  of  stusylng  customer's  oroolem  from  both  that 
of  toe  oeveloper  anu  the  user  in  oroer  to  errlva  at  a 

f'joctlonv’l  'Jeflnltlon  of  system  requl  remehts.  Includes  all 
■Jctlvltles  releteo  to  anelyrinn  nnJ  developing  a clear. 

'jnegulvocal , ana  mituall'/  agreed  uoon  set  of  functional 

f?cecl  ft  cations  for  a oro)sct. 

1 icul  re'jnts  -ma  oeslgn  soecl  flcetlons  (012) 

Jreclse  sped  feat  Ions  of  the  ooeratlonel  characteristics  of 
t'M  final  product. 

:ic'jlre'".»nts  language  (003) 

' conrjter  pronre"  usee  to  orovlPe  a succinct  ana  unanhlguous 
scarification  of  the  systan,  then  con^iter  reoulr enents.  It  rnore 
precisely  allows  reeuirenents  to  Pa  connunlcatad  and 
trensletaJ  In  a hlernrchlcal  mnner. 

(O'ltin?  (001) 

j.i'llest  group  of  conpllahla  code. 

C ;i.-  cenentor  (003) 

' coneuter  orogren  used  to  search  a teoe  to  provide  absolute 
jrogran  locations  In  ni-'ir/  that  ere  relative  to  prograo  labels.  In 
cujitlon,  the  co-"cut9r  ,croir3~  orovloes  storage  ocarsss 
ino  olnary  scaling  associate J 'Ith  user-soecl flea  variable  nanes. 

generator  Is  usao  typically  to  r>r»59nt  a picture  of  a 
jjlectau  portion  of  nenory. 

Cchajula  nnnagenant  (012) 

- 'athou  of  ueter.hlnlrij  vhat  v.-ork  hust  be  done  to  proouce  each 
alenent  of  e system. 

^corinc  program  (003) 

conouter  orogran  that  oerforns  the  sane  calculations 
as  the  target  system.  Vne  program  l.ncludas  a compare  caosblllty 
so  that  results  co.noutso  by  tha  target  system  can  be  automatically 

ca.npared  anu  ol  srreoancles  flagged. 

Sin^jiitor  (003) 

; conoutar  program  that  provides  the  target  system  with 

Incuts  or  responses  that  rase-Pla  those  that  woulo  have  been 

orovUed  by  the  process  for  tha  device  being  simulated.  The 

ci'ulator's  function  Is  to  oresent  uata  to  the  system  at  the 

correct  tine  and  In  an  acceptable  format,  this  Is  a 

category  for  generallzeJ  simulators  tbet  cannot  be  classified 

according  to  soeclflc  functions  under  designators  15,  17,  13,  24,  34,_4V, 

5 3 or  5 1 . 

Snaps  (003) 

i comp'iter  progran  that  allows  Intermediate  oata  valuas  to  ba 
recordad  on  an  external  medium  during  execution.  Snaps  are 
usually  formatted  prior  to  being  recorded  so  that  no  postprocessing 
Is  regulrtd.  Snsps  are  esscmhled  in-Una  with  tha  application  coda 
one  therefor#  cannot  b#  dynamically  overrloden  during  axacutlon 
exceot  for  being  turned  on  or  off. 
softwere  (Old) 

Computer  progran  code  and  Its  associated  necessary  data 
ana  uocunentatlon. 

Software  development  cycle  (OOd) 

i'he  softv.’are  development  cycle  consist  of  four  phases'  definition, 
design,  Inolementatlon  ana  evaluation. 

Software  development  cycle  (014) 

I.  , ragutrements  sped featlon.  Translation 

of  an  operational  (or  apnlicatlon)  ragulrament  Into  a 
statement  of  the  functions  to  bo  performed. 
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2.  Systtn  design.  Tnnsletlon  of  the  requlre-tents  into  e 
description  of  all  the  eeaponents  necessary  to  laplement  the 

system. 

3.  ^rogre^nlng.  Production  of  the  cods  t^lch  will  control 
the  system  and  perform  all  required  logic  and 
conputitlon. 

4.  CnecUMt.  Verl fl eetlor.  ^het  the  coded  and  other 
components  of  the  system  satisfy  the  original  raqui ramants. 

Softn^sra  oeveiopment  process  (OlO) 

The  formal  process  by  Mhieh  project  ohjeetlves  are 
transformed  into  project  requirements,  then  into  design 
...  soecl fl cations,  Inplenentec.'  into  cods,  testeu,  and  finally 
placed  end  Mlntalned  In  operational  status. 

Software  errors  (Oil) 

Any  discrepancy  between  a computed,  observed  or  measured  quantity  and 
Its  trje,  specified,  or  theoretically  correct  value.  Errors  are 
Introduceo  Into  software  by  hunan  olstekes  that  isi  deflclancles  or 

misinterpretations  of  design  criteria,  logical  nlstakts,  r/ntatlcal 
mistakes  made  In  transcribing  program  statamants  into  tht  Input  data, 
software  factory  (012) 

\ collected  set  of  tools,  methodologies,  and  a eonmon  data  base 
thpt  provide  a proctaural  approach  to  the  successful  eonpletlon  of 
software  projects. 

Soft'-ore  feilure  (Oil) 

Software  monitor  (003) 

t coTp>jter  program  that  provides  detailed  statistics  about 
s/stem  oerf ornnnee.  ?9cause  softv/ere  monitors  reside  in  memory, 
they  have  access  to  all  the  tahles  the  syster  maintains.  Therefore, 
t*.*y  can  easily  examlns  such  things  as  core  usans,  gutue  lengths, 
InJIvlujsl  program  operstlon,  ano  so  on  to  help  measure  perfornance. 
doft.'ire  pro.iuct  (012) 

.he  software  conponant  of  the  product. 

Software  rellstillty  (002) 

jnffwsr*  reliability  Is  defined  as  the  orosahlllty  that  a 
jiven  loft-'ere  program  operates  for  sonetime  period,  without 
an  external  software  error,  on  the  no  chine  for  v-hlch  It  was 
jeslgnau  given  that  It  is  usac  v.ithln  design  limits, 
ioft-are  rallehlllty  messures  (Oil) 

rrobahl  11  ty  ■'fosuras  of  the  duality  with  which  oaslt^  requirements  have 
nten  transformwe  Into  software  pronroms.  A tyolcal  measure  Is 
•.na  mean  tine  to  failure  as  a function  of  operating  tine, 
ioft  'ore  system  (012) 
iet  ioft'.'cre  Product, 
software  system  scpenoabl llty  (Oil) 

ihe  prp..ablllty  that  the  aopllcatlon  program  together  with  Its 
s'jaervispry  oroprtm,  oats  bese  ana  hardware  will  perform  In  its 
intendso  environment.  The  environment  will  incluoe 
anonelies  anc.  fallurat,  such  as' 

I.  Oaf Icl ancles  In  requirements 

I.  Software  design  errors  (Ineorraet  algorithms,  wore  length 
problems,  timing  problens,  ate) 

J.  Software  failures 
w,  Processor  errors 
i.  henory  errors 

t.  rallures  In  the  communication  network 
V.  r^ilin-ot  In  perlpharial  devices 
w.  operator  nlstekes 
V.  Power  failures 
t-j.  Environmental  failures 

11.  Jrapuel  erosion  of  the  oate  base 

12.  :(aro-.-sre  saturation  (CPU,  aienory,  I/o  cbennela) 

Software  tasting  (Oil) 

I'he  process  of  exerclelng  software  in  an  attempt  to  detect  errors  which 
exist  in  the  coda.  Software  testing  ooes  not  orove  that  a program 
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Is  correct. 

oourcs  u'llt  ortJ  d.it«  (010) 

I'n*  Jste  of  the  last  uoaate  nade  to  the  unit  of  source  code, 
tiourcs  unit  start  date  (013) 

ine  Jste  the  unit  of  source  code  was  entered  Into  tho  ?SL. 

*o*cl f : cat! on  (012) 

in»  technical  definition  of  the  system  and  Its  parts. 

Sped  fl  cation  omission  error  (013) 

error  for  a unit  of  source  code  associated  '.^Ith  a 
prov^ren  oue  to  onlsslons  in  the  program  specifications, 
jceclflea  oerfornance  re^ul renants  (002) 

A irltten  reoulrenant,  fl'jure  of  nerlt,  or  parameter  which 
Tiall tatlvely  or  quantatl tl vely  daflnss  system  performance. 
jt*ncarcjs  003) 

.->roca  Juras.  rules,  ana  convantlons  used  for  prascrlbln;;  disciplined 
aro^ram  ueslon  (oro-;r.»m  structurin'),  an  I data  structuring)  and 
I .:plemsntstlon.  Architecture  and  nartl  tlonln'j  rules,  documentation 
convantlons,  conf Iquratlon  and  data  mamoanent  procedures,  etc., 
ere  enenj  those  standards  to  be  disseminated, 
btar.jnr'.s  enforcer  (033) 

coipiter  orooram  used  to  autonetlcally  determine  whether 
prescribed  pro>7rn!rmln^  standards  and  practices  have  been  adhered  to, 
jho  program  can  ch.ec'c  for  violations  to  standards  sot  for  such 
conventions  as  program  size,  commentar/,  structure,  etc, 
atetsnents  (OCI) 

cro^rammlnn  len^ia^e  at  tho  source  code  level, 
j'catlstlcal  ore-jictlon  (OOV) 

iU  computation  of  i confidence  factor  thet  Indicates  the  effectiveness 
of  the  oro^rammlnj  ano  verification  process  by  Inserting  errors 
Into  the  soft'-<are  system. 

-cap-.’lsa  raflnanent  (304) 

ine  process  of  deflnlnj  data  In  more  and  more  detail  as  tho 
nasd  arises  Jurlnn  the  pro^rarmlng  process, 
structured  pro-nram  (i305) 

Program  constricted  of  a basic  set  of  control  logic 
figures  v.’hlcm  provide  at  least  the  followlngi  sequence 
of  two  operations,  conditional  branch  to  one  of  two  operations 
anu  raturn,  ana  reoetltlan  of  an  operation,  A structured  program 
has  only  one  entry  ano  one  exist  point.  In  aadltlon,  a path 
»lll  exist  from  the  entry  to  each  nooe  and  from  oach  node  to 
the  exit, 

btructurel  program  seyaents  (012) 

' combination  of  program  stops  ano  calls  to  lo-ver-levol  program 
segments. 

structured  programming  (003) 

»ne  technique  used  In  structured  orogramnlnj  (limited  number 
of  logic  structures,  top-uown  development,  etc.)  make  It  easier  to 
jtvelop  and  verify  completed  computer  programs.  Aids  that  help 
immlement  these  techniques  follow. 

1.  Automated  language  restructuror  - a computer  program 

used  to  restructure  source  programs  into  an  acceptable  structured 
form. 

2.  Language  enhancer  - a computer  program  used  to  provide 
existing  hlgh-^rder  language  compilers  with  acceptable 
language  constructs  for  structured  programming.  Language 

an nan  ears  Include  both  the  preprocessor  and  nacro  maehanlzations 
nresently  In  ust. 

3.  So'jret  coda  tndanter  - a eomputar  program  used  to  autonetlcally 
indent  source  cooe  listings  to  add  to  their  readability. 

a.  Vrpductlan  support  libraries  > a fom  of  production 
librar/  used  to  record  end  store  programnlng  data, 
structured  prograir'’lng  (005) 

(J:')  the  process  of  oaveloolng  stnctured  programs* 

^ssoclated  with  structureo  orogrannlng  are  certain 
practices  such  as  Indatatlonsof  source  coda  to  raprasant  logic 
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Itvtltf  tht  ust  of  IntflllQint  cut*  na-sts  ona  dtscrlptlv* 
eonMntory. 

Structurao  programing  (0I2> 

A prograrning  dlaclpllna  providing  a naans  of  txprtssing  a cystan 
wtslgn  t^at  ansurts  a tastobla  and  unoarstandabla  inplanantatlon 
and  that  anforctt  slnplt  ana  trall-dtf inad  eorractlons  batwaan 

progra-n  nodulas.  Tha  progran^lng  dlsclollna  utas  tht  rapaatad 
application  of  a anall  numbtr  of  basic  control  atata.tants  to 
forn  slnplt  progran  constructs  that  raprtsant  largt  and  cenplax 
prograns.  Tha  procast  of  craatlng  tha  progran  nooulat  Includat* 
•Btlng  local  or  tactical  pronramnlng  daclslons  within  tha 
dtslgnao  nodule i vrltlng  progran  segnents  that  raprcsant  thasa 
decisions  I ano.  Integrating  program  sagnents  Into  a unit 
corresponding  to  a aysten  nodule. 

Structured  progr^nnlng  (015) 

dtrjcturad  programing  Is  strictly  nora  of  a program 
organization  olsclollna  than  a design  tachnlgue,  but  It 
can  be  used  to  significantly  anhanca  nost  of  the  available 
design  tachnlpuas.  It  It  basically  a sat  of  standards  for 
organizing  tha  control  structure  of  a sat  of  computer  progrant, 
i'na  key  loaas  in  structured  programing  arat 

1.  Only  "oropar  programs"  having  ona  flow  of  control 
in  ana  out  of  each  unit  shoulc  be  davalopad. 

2.  Only  three  basic  control  structures  are  allowedi 

D ):;ilL^,  IrrncVfLSS,  and  SHTJE'.'CH.  fhese  three  structures 
are  sufficient  to  express  any  oroper  progrom(l5).  Tv/o 
acJitlonal  structures  are  ootlonal,  the  DOV'filL  end  the 
CksE. 

j.  Programs  are  organized  according  to  a hierarchical, 
mojulpr  sloc>  structure. 

■i.  Additional  standards  are  Imposeo  on  the  slxt  of  medulas, 
the  forr.attlnr.  of  Instructions  and  deelarotlons,  program 
co-mentary,  ate. 
dP5hm,  jsrry 

structured  programming  languere  (012) 

A lan7ja;e  that  suooorts  ihc  concepts  of  Structured 
Programming  and  narmlts  closer  integration  of  menegement  ano 
production  tools. 

Structured  programming  technology  (005) 

A term  rt^lch  collectively  references  the  foliovlng  lliti 

1.  Program  desfmn  languaoe 

2.  Programming  support  library 

J.  1*00  cown  structured  nrogrammlng 
Structured  programs  (012) 

snail  computer  programs  written  in  a restricted,  ilsple  syntax, 
structures  eegmant  (005) 

A logically  complete  .set  of  executable  Instructions 
comitnjeted  of  nested  structwrad  orogran-.ing  figures.  In 
addition  to  or  In  piece  of  exeeutahla  Instructions,  o 
struetureu  segment  may  Include  nontax eeut able  Instrunentlons 
such  as  osta  daelarstlons  anu  descriptive  commentary. 

Structured  sourqa  code  listing  (005) 

A listing  for  a top  down  atrueturad  program  (TOS?) 
eo->prlssd  of  tht  following  aactlons* 

1.  saztlon  I contains  tht  first  asecutable  structured 
segment  (eomnonly  referred  to  ss  the  too  level  segment  as 
coJto  In  tha  aouret  programming  lenguaga. 

2.  Section  2 conuins  all  rareinimg  strueturad  aagmants. 
the  structurao  tagnents  are  a lohabatl zed  by  name.  As  In 
Section  i each  structured  segment  la  rtpraeantad  ss  coded 
in  tha  Bourea  programr.inn  lah7jags. 

d.  Section  3 contains  the  axscuting  seduenea  among  the 
strueturad  segnants. 
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itrucfjrsJ  v.a  li:-thro'joh  (009)' 

\ '3'»'?rle  nan#  ^lv^n  to  ^ s«t  of  t#chnlnu3S  (I.#.,  clesl-jn 
V'llljjtion,  assign  v#rl  flection,  #na  cod#  verification),  each  with 
jlffor^nt  o&jjctlvea  ono  #>icn  occurring  -it  different  tines  in  th# 
sdrtwar#  oeviloorent  cycl#. 
j'j&syjtsn  (On) 

t cnJlvlslon  of  a softwere  systsT  with  a neenlnoful  product 
to  the  user  and  Is  comrlssd  of  one  or  nor#  pro<;rans.  A 
sd'.syst#"  nan#  lo#ntlfl#s  th#  to?  unit  of  a tri#  structiir*. 
j/stet  (Oli) 

Consists  of  lore  then  one  software  subsysten  ana  provides  a 
solution  to  3 prohlen. 
i/ste^  ocslr.n  (012) 

I'-e  orocess  of  transferring  the  oesl^n  reiulrenants  Into  an 
o'/erill  cysten  stricture  that  suoonrts  proorans  satisfying 
t'os#  r''"')! repents.  Inclucss  ell  ictlvltlis  concerned  with 
tr’os  fonlng  functional  reoul  renents/soecif  Icitlons  Into  a 
0 trjct'ira J Programlni  systen  caoable  of  satisfying  those 
re  ••ilrn“.?nts. 

Cystei  sl“iletlons  (003) 

Jo-ojfr  syst#"  slnuletlon  Is  a technliue  us#;)  to  predict  system 
P’rfor'ans#  by  exercising  e ncdel  of  the  system  harc-.;ire/ software 
ov'r  tl"e.  ol'iilstion  cssea  on  well-olenned  ovoerlnents 
ro oresentatlve  of  the  re-3 l--«)rl d cnvlronnent  will  proouc# 
results  that  heln  verify  and  innreve  systsh  oerfornance. 
i •-  slhulatlons  are  also  usso  to  help  orsJlct  how  the  system 
■111  ret’ct  to  alternative  loaos  with  -■oJlfle-J  cohfl'gurotlons. 
-.ioclflc  language  systans  such  as  aCtiS,  C53,  o'CS:(T,  and  SA'l 
have  anch  devlsecj  to  act  es  aids  to  Implementation. 

-yste-.  verification  (012) 

Ml  activities  cohcarnao  with  ascertaining  that  th#  system 
oirfons  es  tha  customer  Intenoed. 

4-:\C:-'  -.rogrem  (003) 

'i  coi?-jt»r  prograw  chat  recon'.'s  the  chronological  sequence 
or  avents  ta'ton  cy  a targat  orogrem  v-urlng  its  exscutlon. 
las't  -llaston-a  (012) 

A major  visible  event  or  Interfec#  during  3 project. 

;er"'iaal  simulator  (003) 

A co'iTitar  program  uses  to  present  Inpjt  messages  to  the 
control  program  so  is  to  sjoear  that  they  hao  been  Inout  from 
an  aetjai  tarnln-cl  devlc*. 
iist  (003) 

Any  orogm  or  oroceJur#  that  Is  deslgnao  to  obtain,  verify,  or 
orovla#  data  for  th#  evaluctloni  research  anu  development 

(other  than  laboratory  experiments) • progress  In  a ccomol Ishlng 
jv/elop'^ent  ohjactlves*  or  parforrance  ans  operational 
ccoablilty  of  systens,  s'Jbsystems,  components,  and  equlpmants 
i te-'s  ( Afi!  SO-14) . 
lest  j?ci  (003) 

' test  site  that  either  contains  the- actual  haraware  and 
Interfaces  (hardware  test  bad)  or  simulates  them  (software 
test  beo), 

1.  (aru-ware  test  bed  • Includes  actual  computer  and  tntsrfaca 
'.•ry.'ere,  thus  permitting  aetuil  chec'<out  of  hsrJwa re/software 
later  faces  and  actual  Inout/output.  Pme  program  execution  Is 
conrlrmea  using  actual  herdware  timing  cheraeterlstles, 

?'it  the  output  Is  limited,  end  It  has  limited  Jlsgnostle 
ceecbllltles. 

2.  Ooftware  test  bad  - uses  an  Instruction  simulator  to  simulate 
actual  hardware.  The  approach  poorly  represents  actual 

I/O,  pjns  7 to  15  times  real-tlms,  and  Is  an  expensive  method  of 

eohUuctlng  lengthy  testing  of  software.  The  approach  oermlts  full 
control  of  Inputs  end  computer  cheraeterlstles,  allows 
processing  of  Intarmeolatt  outputs  without  destroying 
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r«9l  ttiMi  ana  alleyrs  full  itst  rtatat^bl llty  snj  ola7>ostles. 

Test  oata  (012) 

Xast  artyironr^ant  oata  that  art  praparacf  manually  or  by  an  autonatte 
tast  east  qantrator. 
last  tnta  basa  (009) 

Collactlon  of  oata  storab  on  a eo-mutar  perlohanl  davlca 
(a.7«<  tapa,  tfisV  that  elotsly  antchas  tn;  "raal*'  data  basa). 
Idaally,  a tait  data  base  tnould  ba  Icantical  to  a raal  data 
bast  hut  usually  It  only  providas  raprasantatlvt  data, 
last  arlytrs,  scrlots,  data  otntrators  (002) 

To  run  tests  In  a controlled  ranner.  It  is  often  necessary  to  work 
within  the  franev’orV.  of  e “seenarlo“--e  description  of  a oynanic 
situation.  To  acconollsh  this,  it  is  necessery  to  load  the 
inout  data  filet  for  the  systen  with  data  yaluet  representing 
the  test  situation  or  events  tn  yield  recorded  data  to 
evaluate  against  evoecteo  results,  i'hset  aids  psmit  relatively 
easy  generation  of  data  in  external  form  to  be  entered  autonati celly 
into  the  systen  at  the  proper  tine. 

Vest  nlan  (003) 

t nanagenent  docunent  that  describes  how  and  «enen  specified  test 
oojectlves  will  be  net  (AF!i  8C^I4). 

4cst  olan  docunent  (012) 

A plan  that  tests  the  effectiveness  of  the  object  systen,  as 

Inolenentad,  ano  oetemlnes  ho>'  it  can  be  validated, 
verlfleo,  and  certified.  This  docu'^ent  is  input  to  the 
s'jsseruant  functions  of  the  tiunllty  Assurence  process  ana  to  the 
.-’rogran  VelUetlon  function. 

4cst  result  processor  003) 

A ronouter  prograr  used  to  perfor"'  test  O'ltnut  d'.te  rectuetlon, 
for-'-'ttlng  end  printing.  Sons  perforr.  stetlstlcal  analysis 
..here  the  orlnlnel  date  nay  ce  the  output  of  a monitor, 
iestlng  OC9) 

ensures  hov;  well  the  soeclfi cations  ere  net. 
uext  i'tJ  012) 

rTogren  do:unent''tl on  that  is  praoered  nanuelly  end  updated 
0/  a text  editor. 

lining  amlye.er  (002) 

conpiter  erogr»r,  that  nonltors  end  orlnts  execution 
tine  of  all  orogran  elements  (functions,  routines,  and 
s ■.•.-.routines ). 
lOe  .'o-’.n  jeil.gn  (07) 

I'.en  uo.'.n  design  ir.olles  an  ortarlnn  te  the  sequence  of 
jeclslnns  •••rnich  are  ngj*  m the  •jeco-nnsltion  of  a software 
s/ste  1,  Ly  Paglanlng  ••ith  a si‘'olt  ueserlPtlcn  of  the  entire 
orccess  (toe  ievel)^  Through  a succession  of  refinenents 
ef  vnnt  hes  been  oeflnee  at  tech  level,  lo/er  levels  are 
".necl  f 1 od. 

.00  do-.r  -erorranTlng  (OOo) 

ine  concept  of  nerforninn  in  hierarchical  se.guence  a 
uetelled  oesinn,  code,  integretlon  anu  test  as  concurrent 
0 eerstl  ons. 

To?  udvn  aroyrarniniy  nroctss  (017) 

*.n  expa.nsion  of  functional  sotclf Ications  to  slnplier 
an.)  sl'eoltr  functions  until,  finally,  stetene.nts  of  the 
progranrlnn  lannuoge  Itself  ere  reached, 

.sef  H.  "ills,  "Ton  Dov.'n  Progrennlng  in  Large  S"stens'*, 

" Jatupglng  faehniques  in  Large  Systems",  jJ.  Austin,  (ed), 
r*rentice-.h.ell , 'U  Iv72,  4 1-55. 

Top  JO*^'n  structured  progren  (005) 

(I'JS?)  a ttructureo  progran  with  the  aJwitlor.al  characteristics 
of  the  source  eeae  being  logically,  but  not  necessarily 
phvsieslly,  seg**cnte.d  in  a hltrarchlcal  ninner  and  only 
uepandant  on  coda  alraaoy  writtan.  Control  ef  execution  between 
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Is  rsstrictad  to  transfers  between  adjacent 
Tlerarchlcel  se-i'ints. 

Tod  ^o-in  str'icturoo  oro jr a^^^ n^^  (005) 

»*ie  oncoss  of  developin'!  toe  down  stpjctjred  proqrans. 

•vosoclated  wltn  too  oov.-n  structured  pro  ;r  seinlno  ore  certain 
orectlces  such  as  Injcntitlons  of  source  code  to  reoresent 
locle  levels,  the  use  of  Intelligent  oete  r.acas  md 
uescrlJtlve  co.'nantery.  Vop  aov.-n  structured  orojranslnq 
requires  top  oown  oroqraonln^  as  the  prlnery  lipl enentatlons 
nithojeloiy/. 

Too  jo  'n  testing  (017) 

If  nouules  are  produced  In  a top  down  order,  then  top  down 
testlm  can  also  oa  enoloyeo  uslnn  s partially  coeiplatad 
systs-t  in  which  lower  levels  are  represented  by  proqran 
"stujs",?nd  pronra"'s  are  sxecutaci  in  the  environ, tent 
In  w-.lc.h  t*'ey  will  ectunll/  ooerate.  This  approach 
ill0''3  for  earlier  Integration  and  testing  './hlch  should 
uncover  oroolsns  sooner  then  conventional  tastln<j 
/hare  Inteqrstlon  Is  the  last  step. 

Top-uo'n  nrojran  Jeve looiant  (012) 

iha  orocess  of  coolnj  an'J  testlnj  Structured  Proqrans  successively 
jo'.'n/nra  fron  the  top  level  of  oroqran  dosiqn  to  the  bottom 
levels,  continuously  axerclslr.n  the  sctuil  interfaces  betv/een  oroqram 


-'OQules.  The  hottnn  level  nay  be  standard  oaebaned  routlnes“data 
access  •'ethous,  sort  routines  or  interfaces  with  other  systens. 
i'nls  eoproach  Is  tha  ooooslte  of  the  uouel  one  of  checJclnq 
tne  -otton-l-evel  ”oJules  first  and  wor'tlnq  uo,  finally  Intaqratlng 
cna  testlnq  the  entire  system.  !Jere,  the  integration  of  nodules 
is  2 continuous  process. 

Tracer  oroqra.n  (007) 

i.  oroireo  antlysls  tool  uhlcn  will  enalyie  conouter  oroqrans  loo’clnq  for 

coejo"  - l.e.,  code  In  a oroqren  which  can  not  be  executed. 

Trwnslator  (003) 

\ co’^outer  propran  written  to  acceot  Information  from  one  system 
of  renresentatlon  and  convert  this  Into  equivalent  Information  In 
anothar  system  of  rsorossncatlon. 

Traps  (C3J) 

\ technique  In  !>ilch  the  logic  flow  of  e orograie  Is  Interrupted 
for  the  purpose  of  setting  eslde  interim  results  for  test 
ceasurenent  or  perfornlm  soecial  test  functions, 

Jnlt  (Oil) 

a name'J  subdivision  of  a orogran  'which  Is  capable  of  being 
stored  in  a PSL  and  manlpuleted  as  a single  entity. 

'.'hits  consistency  analysis  prooran  ((X)3) 

*.  comptitar  program  that  analyzes  source  code  to  verify  units 
consistency  for  each  usage  of  each  parameter. 

User  (301) 

The  Injivlduai  st  the  man/nachtne  Interface  who  la  ao.oiying  the 
software  to  the  solution  of  a oroblan,  a.g.  test  or  operstlons. 

Utilities  (.303) 

3T.iputhr  programs  smoloyad  by  other  V/V/C  aids  to  provide  soecial 
services.  These  services  include  preparing  program  aec'<  listings, 
creating  load  tapas.  and  plotting  output  results. 

Utilities  (012) 

Tools  that  fecllltate  tha  production  and  control  of  softwara 
systems.  Thoss  Include  tools  for  data  set  management,  program 
uevalopnent,  and  program  axaeutlon. 

Vslluatlon  (009) 

Tha  process  of  detemlntng  whather  axacutlng  the  aystan  (l.a. , 

software,  hardware,  user  procaduras,  parsonnaU  in  a user 
anvlronmant  causes  any  ootratlonal  difficulties.  Validation 
is  mors  difficult  than  the  verification  proeass  since  It  Involves 
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questions  of  th#  eo^oltt«''J*s  of  tht  iptelf le»tlon  nnJ  tnvlronnent 
Inforntlon.  "ihcre  are  both  iwnuol  ana  eosputsr  baiad  vallOatlon 
ttchnlquts. 

Vall'intlon  (012) 

Th#  proatss  of  tnaurln^  that  specific  pro^ran  functions  nest  their 
detalleo  oesl;n  requirement  spsclflcstlons.  Also,  see  pro^ron 
Vellustlon,  Pro.^ren  VallOctlon  Tools,  sno  VellJetlon  and  Debugging 

Tools. 


Valljatlon  anJ  debyqnln^  tools  (012) 

iPols  relstea  to  the  production  of  cerrsct,  serviceable  orograns. 
Valuation  incluJts  the  prevention,  ostectlon,  diagnosis, 
recovery,  enj  correction  of  srrors.  Debugging  Includes 
correcting  errors  of  both  a logical  and  a elerleel  nature. 

Vsllantlon  criterl.i  (012) 

\ guloe  defining  v;net  will  be  useu  to  determine  eonpletlon  of  a 
nliestonc.  This  Incl'jues  successful  o.oorstlon  of  a tsst  tool, 

or  acceotence  of  a docunent,  or  approval  by  the  review  board. 
v'allJstlon  tools  (012) 

See  V’llJstlon  sno  Debugalng  tools. 

Verification  (03V) 

r-ie  process  of  tieter'-lnlng  whether  the  results  of  executing  the 
soft-are  prouuet  in  s test  envlronnent  agree  vlth  the  specif  1 eotl ons. 
Verification  is  usually  only  concerned  with  the  softyrare's  logical 
correctness  (l.e.  sstlsfylng  the  functlonel  regul resents)  and 
■ey  he  a manual  or  e conpuier  has*;!  process  (l.e.,  testing 
soft'or#  by  executing  It  on  a computer) . 

Verification  (CI2) 

The  process  of  ensuring  th't  the  syst?""  ano  Its  structure 
meet  the  functlop.el  requirements  of  the  2sscllni  Specification 

hocu-ent.  Also  see  Pyste-  Verification, 

Ver 1: Icrtlon  valluatlon  certl flcetlon  (003) 

(of  computer  oroTsms).  foe  nroc»ss  of  determining  that 

tne  computer  oro-rem  was  t'evelopea  In  eccoramnee  ».’lth  the  stated 

3-ecIfl cation  and  setlsf ector ily  cerforms,  in  the  mission  environment,  the 

fuhctlond)  for  ••■.•hi  cn  It  was  ceslgnco  ('u=ri  30>I4>,  See  conputer  progran 

V irif Icatlon,  eo"puter  progra"  validation,  computer  program 

c ‘rtl  fl  cation. 

.‘ersiem  -sllflcatlon  level  (0(3) 

‘..'I  im.'lcatlon  of  the  version  and  mndlflcstlon  level  of  a 
unit  of  eourca  coos. 
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•ASEUNITS: 

Quintiry 


Unit 


iMttlt 

BMI 

UOM 

•lactnc  cuiTMi 
thatmodyMmic  ttmpmiuft 
•mount  luteMnca 
lumlnoui  lnt»n*tty 

SUTPLEMCNTAKY  UMTS 

plant  anflc 
solid  anile 

OOUVED  UMTS: 
Acceleration 

activity  (of  a ladioactivt  touice) 
•n|ular  acceltrslion 
tn|ular  velocity 

tree 

density 

electric  capecitence 

electricel  conductance 

electric  field  strength 

electric  inductance 

electric  potential  difference 

electric  resistance 

electromotive  force 

energy 

entropy 

force 

frequency 

illuminance 

luminance 

luminous  flux 

magnetic  field  strength 

magnetic  flux 

magnetic  flux  density 

magnetomotive  force 

poster 

pressure 

quantity  of  elec'rinty 
quantity  of  heat 
radiant  intensity 
specific  best 
stress 

thermal  conducntitt 
velocits 

viscosity  dvnenic 

viscosity  hinematic 

voltage 

volume 

wavenumber 

work 


metre 

kilogram 

second 

ampere 

kelvin 

mole 

candela 


radian 
stared lan 


metre  per  second  squared 

disintegration  per  second 

radian  per  second  squared 

radian  per  second 

square  metre 

kilogram  per  cubic  metre 

farad 

siemens 

volt  per  metre 

henry 

volt 

ohm 

volt 

loule 

loule  per  kelvm 

nevtion 

hertz 

lux 

candela  per  square  metre 
lumen 

ampere  per  metre 

weber 

tesla 

ampere 

watt 

pascal 

couiorrb 

louie 

watt  per  steradian 
lOule  per  kilogram-kelvin 

pascal 

watt  per  metre-kelvin 
metre  per  second 
pascal-second 
square  metre  per  second 
volt 

cubTc  metre 
reciprocal  metre 
loule 


Formula 


rad 

ar 


fIVS 

(disintegration  )rs 
rads 

radrS 

m 

kgm 

A-sV 

s 

AV 

V'm 

H 

Vs  A 

V 

V\  A 

V A 

V 

W A 

1 

N-m 

l-K 

N 

kg-ms 

Hz 

(cycle)s 

lx 

Imm 

edm 

Im 

cd-sr 

Am 

VVb 

V-s 

T 

Wb  m 

A 

W 

Is 

Pa 

N m 

( 

As 

1 

N-m 

Vt  sr 

Ikg  k 

Pa 

N m 

V\  m-k 
m s 

Pa-s 
m s 

\ 

W A 

m 

(wavel  m 

1 

Nm 

SI  ntenxEs 


Mulliplit  anon  Factors 

I'refis 

Si  Symii 
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M 
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k 
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h 

to 

10’ 

daks* 

di 

0 1 

10-' 

deci* 

d 

0 01 

10-' 

ranti* 

1 

0 001 

in-' 

mllli 

ni 

u non  001 

10-* 

micni 

M 

0 OtNI  INIU  (101 

111-* 

nano 

n 

0 000 IKKI  non  001 

10- '• 

pii.o 

tamln 

}’ 

0.000  000  000  INKI  001 

to-'* 

1 

0 000  000  000  non  imu  ooi 

10- •• 

ettu 

• 
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MISSION 

of 

Rome  Air  Devebpmenl  Center 


JWDC  pleins  Uid  conducts  research,  exploratory  and  advanced 
development  programs  in  command,  control,  and  communications 
(C^I  activities,  and  in  the  areas  of  informatior.  sciences 
and  intelligence.  The  principal  technical  mission  areas 
are  coimunications , electromagnetic  guidance  and  control. 
Surveillance  of  ground  and  aerospace  objects,  intelligence 
data  collection  and  handling,  information  system  tectmology, 
ionospheric  propagation,  solid  state  sciences,  microvava 
physics  and  electronic  reliability , maintainability  and 
compatibility. 


